Transformer block — the architectural Lego

📅 Day 53. The transformer block — the Lego brick that stacks to build every modern LLM.

🏗 The block has two sublayers. Each sublayer is wrapped in a 'pre-norm + residual' pattern.

📐 Sublayer 1 — Multi-head attention.
→ LayerNorm the input.
→ Multi-head attention on the normalised input.
→ Add the result back to the original input (residual connection).

📐 Sublayer 2 — MLP (feed-forward).
→ LayerNorm the result of sublayer 1.
→ Two linear layers with GELU in between (typically inner dim is 4x outer dim).
→ Add the result back (residual connection).

That's it. One block has these two sublayers. Stack 12, 24, 48, 96 of them, add an embedding layer at the bottom and an output head at the top, and you have GPT, Llama, BERT, Claude.

🌐 Same skeleton; different scales:
→ BERT-base — 12 blocks, hidden dim 768. ~110M parameters.
→ GPT-3 — 96 blocks, hidden dim 12288. 175B parameters.
→ Llama-3-8B — 32 blocks, hidden dim 4096. 8B parameters.
→ Llama-3-405B — 126 blocks, hidden dim 16384. 405B parameters.

🧠 The two simple sublayers, plus residual connections and layer normalisation, are what make 100B-parameter models trainable. Residuals carry gradient through deep stacks; LayerNorm keeps activation magnitudes stable.

🔧 Modern variations (post-2020):
→ RMSNorm replaces LayerNorm. Simpler, slightly faster, same effect.
→ SwiGLU replaces standard MLP. Slightly more parameters, better performance.
→ Rotary positional encodings (RoPE) replace absolute position embeddings. Better extrapolation to longer sequences.
→ Grouped-query attention (GQA) reduces inference memory. Used in Llama-2/3.

All these are optimisations on the same core block. The 2017 design holds.

💡 Master the block. Stack to taste.

🤔 Same transformer block. Different attention mask. Different model.

👁 Encoder (BERT, RoBERTa, DistilBERT) — bidirectional attention. Each token can attend to ALL positions, including future ones. Good for understanding tasks where the whole input is available — classification, NER, embedding.

🎯 Decoder (GPT, Llama, Claude) — causal (autoregressive) attention. Each token can attend ONLY to itself and earlier positions. Good for generation — predict the next token given everything so far.

The attention mask is what enforces this. In decoder, mask out the upper triangle of the attention matrix (so position i can't see position j > i).

🔄 Encoder-decoder (T5, BART, the original 'Attention Is All You Need' transformer) — encoder reads input; decoder generates output, cross-attending to the encoder's output. Used for sequence-to-sequence tasks like translation and summarisation.

📊 Where each shines:

👁 Encoder for:
→ Embeddings (sentence-transformers, BAAI/bge use encoder-only).
→ Classification (BERT for sentiment, intent, topic).
→ NER, QA, similarity.

🎯 Decoder for:
→ Text generation (chat, completion, code).
→ Question answering with reasoning.
→ All major LLMs in 2026 are decoder-only.

🔄 Encoder-decoder for:
→ Translation (T5, mBART).
→ Summarisation (BART, T5).
→ Some niche seq2seq tasks.

💡 In 2026, the trend is decoder-only for almost everything generative. ChatGPT, Claude, Llama, Gemini — all decoder-only. The encoder-decoder distinction matters less than it used to; you can do almost any task with a decoder by clever prompting.

🎯 But for embeddings and dense retrieval, encoder models still dominate. BAAI/bge, sentence-transformers, Jina embeddings — all encoder-only. They're better suited for the bidirectional understanding that retrieval needs.

💻 The transformer block in PyTorch. Pre-LN style (norm before each sublayer) — the modern default.

Look at the snippet.

🏗 Block class:
→ self.ln1 = nn.LayerNorm(d) — first LayerNorm.
→ self.attn = nn.MultiheadAttention(d, h, batch_first=True) — multi-head attention.
→ self.ln2 = nn.LayerNorm(d) — second LayerNorm.
→ self.mlp = nn.Sequential(nn.Linear(d, 4*d), nn.GELU(), nn.Linear(4*d, d)) — feedforward MLP.

🔄 forward(x, mask=None):
→ a, _ = self.attn(self.ln1(x), self.ln1(x), self.ln1(x), attn_mask=mask). Apply LayerNorm, then attention. We pass the same input as Q, K, V (this is self-attention).
→ x = x + a. Add the attention output to the original input (residual connection).
→ x = x + self.mlp(self.ln2(x)). Apply LayerNorm, MLP, residual.
→ Return x.

📊 The shape stays (B, T, D) throughout. No dimension changes inside a block.

💡 Pre-LN vs post-LN. The original Transformer (2017) used post-LN — applied LayerNorm AFTER the sublayer + residual. Pre-LN (LayerNorm BEFORE the sublayer) was found to train more stably with deeper models. All modern transformers use pre-LN.

🧠 The MLP details:
→ Inner dimension 4x outer (d → 4d → d). Standard since the original paper.
→ GELU activation. ReLU also works; GELU is smoother and slightly better.
→ Some modern models use SwiGLU instead of GELU (Llama, Mistral). Slightly more compute, slightly better quality.

🚀 This is the core block. Stack N of them. Add an embedding layer below. Add a language modeling head above. You have a transformer.

💡 You'll never write this from scratch in production. But understanding the structure makes every transformer paper readable.

💡 Pro tip — writing your own transformer is a great learning exercise. It's a terrible production choice.

🚨 Why? Because the production transformer ecosystem in 2026 is mature, optimised, and battle-tested. Hand-rolling means you reinvent — usually with bugs and worse performance.

📦 Use these instead:

🤗 HuggingFace transformers. The de facto standard for transformer fine-tuning, inference, and most training. Supports thousands of pretrained models, all major architectures, multiple training paradigms (causal LM, masked LM, sequence classification).

⚡ vLLM. Best in class for inference serving. Paged attention, continuous batching, OpenAI-compatible API. 10-30x throughput vs naive .generate(). Used in production at many AI companies.

💨 TGI (Text Generation Inference). HuggingFace's inference server. Similar performance to vLLM with deeper HF integration.

💻 llama.cpp. C++ implementation for local inference. GGUF quantised models run on CPU and consumer GPUs. Best for laptops, edge devices, and resource-constrained environments.

🛠 TRL / Axolotl / unsloth. For fine-tuning. SFT, DPO, RLHF — all the modern training paradigms. Configure with YAML; train with one command.

📚 PEFT (parameter-efficient fine-tuning). LoRA, QLoRA, prefix tuning. Fine-tune big models on consumer GPUs.

🎯 What you save by using these:

→ Hardened bugs. Years of issue reports have been fixed in the public libraries.
→ Performance. FlashAttention, kernel fusions, quantisation — all integrated.
→ Compatibility. Read any HuggingFace model; share your fine-tuned model on the hub.
→ Speed of iteration. Hours, not weeks.

💡 Learn the architecture. Use the libraries. Save your time for the parts that matter — data, evaluation, prompts, your specific application logic.

🚀 Don't reinvent. Compose.

📅 End of Day 53.

✅ Recap:

🏗 Transformer block = LayerNorm + Attention + Residual + LayerNorm + MLP + Residual. Two sublayers per block. Stack to depth.

👁🎯 Encoder vs decoder = mask choice. Encoder is bidirectional (BERT). Decoder is causal (GPT). Encoder-decoder for seq2seq (T5).

💻 Pre-LN block in PyTorch. Compact implementation. Modern default.

📦 Don't write your own transformer in 2026. Use HF transformers for training, vLLM for inference, llama.cpp for local. The ecosystem is mature.

🧠 Reflection — the transformer block is one of the most consequential designs in computing history. It powers GPT-4, Claude, Llama, Gemini, every AI assistant you've used. The architecture has barely changed since 2017; what changed is scale (more parameters, more data, more training compute) and engineering (FlashAttention, mixed precision, distributed training). The lesson — small architectural ideas with massive impact when scaled.

🚀 Tomorrow, Day 54 — BERT and the encoder family. Why they still matter for embeddings and small-data classification. The DistilBERT trick for production. Then Friday: GPT-style decoders. Saturday: fine-tune vs prompt vs RAG.

💼 Three days left in week 8. Then RAG, agents, automation, career. The home stretch is more about applications than fundamentals.

👋 See you in the morning.