Multi-step retrieval — when one query isn't enough

Question: 'How does X compare to Y?'
No single chunk has the answer. You need:
1. Retrieve about X
2. Retrieve about Y
3. Synthesise.

Question: 'In the 2024 annual report, what's the YoY growth in segment Z?'
1. Retrieve the 2024 report.
2. Retrieve the 2023 report.
3. Compute.

Multi-hop or 'iterative retrieval' splits a query into sub-queries, retrieves for each, and synthesises. The LLM orchestrates the loop.

The pattern:

1. Send the user query to an LLM with a 'decompose this into sub-questions' prompt.
2. For each sub-question, run normal RAG.
3. Return all retrieved chunks + sub-questions to a final LLM with the original question.
4. Final LLM synthesises.

Latency: 2x normal RAG. Quality: dramatic on multi-entity questions.

Cheap when sub-questions can run in parallel (asyncio.gather). Don't sequence what you can fan out.

Decompose, fan out with asyncio.gather, synthesise. Save round-trips by parallelising. The LLM is the only sequential step at the end.

Multi-hop is 2-3x more expensive than vanilla RAG. Don't apply it to every query.

A cheap classifier (LLM call, prompt-only) decides 'is this multi-hop'. If yes, decompose. If no, single-shot. Keeps your average cost down while handling the hard ones.

Or expose two endpoints: '/quick' and '/research'. Let the user pick. Latency expectations differ.

Day 67 done.

- Some questions need multi-step
- Decompose → fan out → synthesise
- Async parallelisation
- Apply selectively

Tomorrow (Day 68): agentic RAG — letting the LLM decide what to retrieve.