Agent failure modes + week 11 wrap

Real agent bugs I've hit:

1. Tool spinning — calls search() 50 times. Cap turns.
2. Hallucinated tool — invents a tool name not in the schema. Strict response_format + reject unknown.
3. Lost-in-context — gives up halfway. Add periodic 'restate goal' nudges.
4. Confidence collapse — claims success without verifying. Force a check-tool at end.
5. Cost runaway — quietly burns $50 per query. Set a hard cost cap per session.

Each has a fix. None require a smarter model. They require better engineering.

Three eval axes for any agent:

1. Completion — did it solve the user's task? (LLM-as-judge or held-out tests)
2. Cost — total tokens / time / tool calls
3. Safety — any banned actions, unsafe outputs, leaked secrets

A 'great' agent that costs $5/query is not great. A cheap agent that fails 30% of tasks is not cheap. Track all three; reject changes that regress any.

langsmith / langfuse make this easy. Add it to CI for any agentic feature.

Always cap. Cost cap kills runaway loops. Step cap kills infinite ReAct dances. Combine them; hit either; force final answer.

Agents in production sometimes go wrong en masse. A bad system prompt deploy. A regressing model. A novel adversarial input.

Ship a kill-switch from day one:
- Feature flag to disable all agentic flows
- Force fallback to non-agentic path (vanilla RAG, FAQ, human escalation)
- One-click toggle

When things go wrong at 2am, you flip the switch and debug calmly.

Week 11 done. 77 days, 385 posts in.

This week:
- LLM agents defined honestly
- Tool design that scales
- Plan-then-execute
- Multi-agent (used carefully)
- LangGraph as default
- Memory: working, short, long
- Failure modes + caps

Next week: automation. Python scripts that schedule themselves and free your time.