Why RAG — and why it's not just 'context-stuffing'

RAG gives an LLM access to your documents at query time, instead of training the documents into the model.

Flow:
1. User asks a question.
2. System retrieves relevant chunks from your docs.
3. Stuff those chunks + the question into a prompt.
4. LLM answers using both its knowledge and the retrieved context.

Why it dominates 2026 LLM apps:
- Your data stays private (no fine-tuning required)
- Updates instantly (just re-index docs)
- Cheaper than fine-tuning
- Easier to debug (see what was retrieved)

Production RAG has 5 hard problems:

1. Chunking — splitting docs without breaking context
2. Embedding — choosing a model that captures YOUR domain's similarity
3. Retrieval — top-k similarity vs hybrid search vs reranking
4. Augmentation — fitting only relevant context into the prompt
5. Evaluation — knowing if retrieval is even helping

Any tutorial that shows you 'RAG in 20 lines' is solving 1/5 of the actual problem. The other 4 are where most production effort goes.

A working RAG with 4 imports. Reads markdown, embeds chunks, finds top-k for a query, stuffs into a prompt. The starting point — week 10 we make it production-grade.

Bad RAG returns an answer. Good RAG returns an answer + the chunks used.

Why:
- User can verify
- You can debug bad answers (see what was retrieved)
- Builds trust — 'cite your sources'
- Reveals retrieval failures the LLM hides

In the prompt, ask the model to quote the relevant snippets. In the UI, show retrieved chunks as collapsible 'sources'. This single feature ships more confidence than any model upgrade.

Day 57 done.

- RAG = retrieve + augment + generate
- 5 hard problems, not 1
- 30-line naive RAG
- Always show sources

Tomorrow (Day 58): chunking. The deceptively hard step that makes or breaks every RAG app.