Embedding choice — the silent quality lever

Cosine similarity returns a number. The embedding model decided what counts as 'close'.

A general-purpose model (text-embedding-3-small) thinks 'How do I refund?' and 'What's your refund policy?' are similar.

A bad model might rank 'How do I refund?' close to 'How do I sign up?' because both start with 'How do I'.

For your domain, evaluate. Pick a 'small' good general model + a domain test set. Measure recall@5 against ground truth queries. Switch models if you can do better.

MTEB (Massive Text Embedding Benchmark) ranks embedding models across retrieval, classification, clustering tasks.

Use it to shortlist. Top of the leaderboard in 2026 includes BAAI/bge-m3, voyage-3, jina-embeddings-v3, OpenAI text-embedding-3-large.

Caveats:
- Mostly English. Multilingual is a separate ranking.
- Domain-specific tasks (medical, legal, code) may not match average rankings.
- Bigger isn't always better — small models often work fine on specific tasks.

MTEB shortlists. Your held-out evaluation decides.

Hand-curate 30 (query, expected_chunk_id) pairs. Then for each, embed the query, retrieve top-5, check if expected chunk is in there.

recall@5 = number of hits / total queries.

This 30-minute exercise tells you more than 100 papers. Compare 3 embedding models on the same set. Pick the winner.

Embeddings from one model aren't comparable to another. If you switch from bge-small to OpenAI text-embedding-3-large, every existing vector in your store is invalid.

Rules:
1. Pin the embedding model + version in your config.
2. Re-index everything when you change models.
3. Tag each vector with the model+version it came from. Refuse cross-version retrieval.

Teams that skip this end up with mysterious quality drops. Save the headache.

Day 59 done.

- Embedding choice = retrieval quality
- MTEB shortlists; eval decides
- 30-pair recall@k eval
- Pin and re-index on change

Tomorrow (Day 60): vector databases — what they actually do, and which to use.