Agentic RAG — let the LLM decide

Vanilla RAG: retrieve, then prompt. Always retrieves, even when not needed.

Agentic RAG: the LLM decides whether to retrieve, what to query, and when to stop.

Mechanism: the LLM is given a 'search' tool. On each turn, it can call search(query) or answer directly. Loop until it answers.

When this wins:
- Mixed queries (some need docs, some don't)
- Iterative research questions
- Comparisons that span multiple retrievals

Cost: variable per query. Worst case 5-10 LLM calls. Worth it for hard questions; overkill for easy ones.

OpenAI, Anthropic, Mistral, and most open models support tool / function calling natively. You define a JSON schema for the tool; the model calls it; you execute; return result; loop.

The loop:
1. Send messages + tools to LLM.
2. If response has tool_call, run the tool, append result to messages.
3. Repeat until response has final answer.

Agentic RAG = tool-calling LLM + a retrieve(query) tool. 30 lines of Python. We'll write it tomorrow during agents week.

Define a search tool with a JSON schema. Pass to chat.completions.create. The model decides when to call. We loop until done.

This pattern works identically in Anthropic / Mistral / open models — only the API surface changes.

Without a cap, an agent can call search() 50 times on a confusing question.

My default: max 5-8 tool calls per session. After cap, force the model to answer with what it has.

Also log every tool call — query + result. When debugging 'why did the agent give a weird answer', the tool log tells you exactly which retrievals it saw.

Day 68 done.

- Agentic RAG = retrieval as tool
- Tool-calling makes it trivial
- 30-line implementation
- Cap calls; log everything

Tomorrow (Day 69): GraphRAG — when entities and relationships matter more than chunks.