Transformer

The transformer is a neural network architecture introduced in the 2017 Google paper "Attention Is All You Need" that replaced earlier recurrent (RNN/LSTM) architectures as the foundation of modern AI language models. Its key innovation is the self-attention mechanism, which lets the model weigh the relevance of every other token in the input when processing each token — in parallel, rather than sequentially — making transformers both more capable at capturing long-range relationships in text and dramatically more efficient to train on modern GPU/TPU hardware, since attention computations parallelize well across many processors at once. This matters for builders mostly as background context: virtually every LLM you'll integrate with — Claude, GPT, Gemini, Llama, Mistral — is a transformer (or a close architectural variant), so understanding the basic shape explains real, observable behaviors. A transformer processes text through stacked layers, each combining a self-attention sub-layer (which tokens should influence which other tokens) and a feed-forward sub-layer (further transforming each token's representation), with the final layer producing a probability distribution over the vocabulary for what token comes next. A simplified concrete example: given the input "The cat sat on the ___", the transformer's attention mechanism lets the model notice that "cat" and "sat" are highly relevant to predicting the next word, weighing them more heavily than "the," and outputs a probability distribution heavily favoring words like "mat," "floor," or "chair." Architecturally, transformers come in three flavors relevant to different tasks: encoder-only (like BERT, good at understanding/classification, powers many embedding models), decoder-only (like GPT and Claude, good at generation, predicts the next token autoregressively), and encoder-decoder (like T5, good at sequence-to-sequence tasks like translation). Nearly all modern chat-focused LLMs are decoder-only transformers. Knowing this helps explain why LLMs process input non-sequentially (all tokens attended to at once, not read left-to-right like a human), why longer inputs cost more compute (attention cost scales roughly quadratically with sequence length in the original design, though newer techniques mitigate this), and why the same underlying architecture powers both a tiny on-device model and a trillion-parameter frontier model. The transformer's parallelizable design is also precisely why the current AI scaling era became possible: because every token's attention computation can run simultaneously across many GPU cores rather than waiting for the previous token to finish (as older recurrent architectures required), transformers could be trained on far larger datasets in far less wall-clock time — a hardware/architecture fit that directly enabled the jump from small research models to today's massive frontier LLMs. Builders don't need to implement transformer internals themselves (virtually every model is accessed via API or a pre-built library), but understanding this parallel, attention-driven structure explains why prompt ordering and structure genuinely affect output quality, and why techniques like retrieval that reduce irrelevant context tend to improve results.