Attention

Attention (specifically self-attention in transformers) is the mechanism that lets a neural network dynamically decide, for every token it processes, how much "focus" to place on every other token in the input — instead of processing text strictly left-to-right like older recurrent models, a transformer can directly relate the first word of a sentence to the last. Mechanically, attention works by projecting each token's vector representation into three roles — a query, a key, and a value — then computing a compatibility score between each token's query and every other token's key (typically via a scaled dot product), converting those scores into weights via softmax, and using those weights to produce a weighted sum of value vectors. The result: a new representation of each token that's been "informed" by the tokens most relevant to it. Multi-head attention runs this process several times in parallel with different learned projections, letting the model simultaneously track different types of relationships (grammatical structure in one head, coreference in another, topical relevance in a third). This matters for builders because attention explains several practical LLM behaviors: why models handle pronoun resolution and long-range dependencies well ("The trophy didn't fit in the suitcase because it was too big" — attention lets the model correctly link "it" to "trophy," not "suitcase," based on context), why very long documents can dilute a model's focus on any one part (attention weight gets spread thinner across more tokens), and why techniques like retrieval (narrowing what's in context) improve accuracy — less irrelevant content competing for attention. A concrete illustration: when a transformer processes "Paris is the capital of France, and it has a population of about 2 million," attention lets the token "it" attend strongly to "Paris" (not "France") when the model has learned from training data that population figures typically follow the city being discussed, producing a coherent continuation rather than a random guess. Attention is also the component researchers optimize hardest for efficiency (e.g., FlashAttention, sparse attention, sliding-window attention) since its cost scales with sequence length, directly affecting context-window size and inference latency. Modern efficient-attention research (FlashAttention and similar techniques) has made attention computation faster and less memory-hungry without changing its mathematical behavior, which is part of why context windows have grown so dramatically in recent model generations — the same core mechanism now runs over far more tokens at acceptable speed and cost. For builders, the practical takeaway isn't the math but the behavior it produces: models with strong attention mechanisms handle instructions buried in the middle of a long prompt less reliably than instructions placed at the start or end (the "lost in the middle" effect), which is a real, testable reason to put your most important instructions early or late in a prompt rather than in the center of a long context block.