Query rewriting — fix the input before the search

User says 'how do i' (truncated, lowercase, vague). Embedding similarity to a doc titled 'Onboarding Procedure for New Employees' is weak.

Fix: rewrite the query before searching. An LLM call (or a fine-tuned tiny model) produces a fuller, more search-friendly version.

Approaches:
- Expansion — add likely synonyms
- HyDE — generate a hypothetical answer; embed THAT for retrieval
- Multi-query — generate N rewrites, retrieve each, union

Non-trivial gains, especially for sparse/conversational queries.

HyDE = Hypothetical Document Embeddings.

Idea: instead of embedding the user's question, ask the LLM to generate a hypothetical answer. Embed THAT. Search.

Why it works: the answer is more similar in shape to documents than the question is. The embedding space loves answer-shaped text.

Cost: 1 LLM call. Latency: ~500ms. Use cheap models for the rewrite (gpt-4o-mini, claude-haiku) — quality of the rewrite isn't the bottleneck.

Generate 3 rewrites of the query. Retrieve top-k for each. RRF the union. Often better than HyDE for ambiguous queries.

Keep the rewrites diverse — explicit, specific, abstract. The LLM can do this in one call returning a JSON array.

User questions cluster. 'How do I reset my password' is the same query 1000 times.

Cache rewrites by hash(query). LRU cache or Redis with TTL. Avoid burning LLM tokens on the same rewrite.

For production: log every rewrite + outcome. The cache becomes a map of 'queries seen' to retrieval traces — gold for debugging.

Day 66 done.

- Bad query = bad retrieval
- HyDE: embed hypothetical answer
- Multi-query rewrites + RRF
- Cache rewrites

Tomorrow (Day 67): multi-step retrieval. When one query isn't enough.