Technologies
Transformer
Attention-based neural network architecture underlying nearly all large language models.
In the book
Events
-
"Attention Is All You Need": the Transformer arrives, the idea ChatGPT will grow from
In June 2017, eight researchers, mostly from Google, proposed a new way to build neural networks: the Transformer. At the time it looked like a paper about machine translation. Today nearly every chatbot, ChatGPT included, is built on the idea.