Transformers & Attention
The 2017 architecture behind every modern LLM.
- ✓What self-attention does
- ✓Why Transformers beat older models
- ✓The intuition, without the maths
Explanation
The Transformer (2017) is the architecture behind GPT, Claude, and Gemini. Its key idea is self-attention: when processing a word, the model looks at every other word in the input and weighs how relevant each is.
That lets it handle long-range context — understanding that 'it' refers to something mentioned ten words earlier.
Unlike older sequential models, Transformers process all tokens in parallel, which is why they train so efficiently on modern hardware.
When a model correctly resolves 'The trophy didn't fit in the suitcase because it was too big' — knowing 'it' = trophy — that's attention at work.
- • Thinking the model reads strictly left-to-right — attention lets it consider the whole context at once.
Write a sentence with an ambiguous pronoun and explain which words 'attention' must connect to resolve it.
1. Self-attention lets a model...