A transformer processes sequences through layers that combine attention with a feed-forward network. Attention moves information between positions; the feed-forward network transforms the representation at each position separately.
Follow a token through one block
Start with a vector representing a token at its position. The attention operation uses that vector to gather information from other allowed positions. A residual connection adds the attention result to the existing representation, rather than replacing it entirely.
A feed-forward network then transforms each position’s vector. Its weights are shared across positions within the layer, but its computation at one position does not read neighboring positions directly. Another residual connection carries the result forward. Normalization helps control the scale of the intermediate values. Its placement differs across transformer designs; the card shows one common block diagram.
Stacking blocks repeats this process. The blocks have similar structure, but they do not normally share the same learned weights. Later layers receive representations that already contain information mixed by earlier layers.
The original transformer and a decoder LLM
The 2017 transformer was an encoder-decoder model for tasks such as translation. The encoder processed the input, and the decoder used both causal self-attention and attention to encoder outputs.
Many generative LLMs instead use a decoder-only stack. A causal mask prevents each position from attending to future tokens. The final representation feeds an output projection that produces scores over the vocabulary. Sampling or another selection rule chooses the next token from those scores.
Parallel training does not mean parallel generation
During training, the target sequence is already known. The model can calculate predictions for many positions in parallel while the mask hides future information. During ordinary autoregressive generation, the next input depends on the token just selected. That creates a sequential loop.
The KV cache saves intermediate keys and values so the decoder can reuse earlier work inside that loop. It changes the work required per step, not the fact that ordinary decoding produces tokens sequentially.
Source: Vaswani et al., Attention Is All You Need. For the component calculations, start with query, key, and value vectors and self-attention.