0
Postmortem prompt for analyzing a production agent failure
Help me write a postmortem for an agent failure in production: failure_description. Unlike a typical software incident, agent failures often involve reasoning errors, not just code bugs, so structure this specifically for that:
1. Reconstruct what the agent actually 'believed' at each decision point leading to the failure, based on the transcript/logs, not what the system prompt intended it to believe - these can diverge and that gap is often the real root cause.
2. Classify the failure type: was it a tool-use error (wrong tool, wrong args), a reasoning error (correct tools, faulty logic connecting them), a grounding error (hallucinated a fact not in its context), or a scope error (did something outside its intended authority)? Pick the primary one, don't hedge across all four.
3. Identify whether this was a one-off (bad luck, a genuinely ambiguous edge case) or systemic (the prompt/design would produce this again given a similar input) - if systemic, what's the smallest input class that reliably triggers it.
4. Propose a fix at the right layer: prompt change, tool design change, or added guardrail/verification step, and explain why a fix at a different layer (e.g. 'just tell it to be more careful') wouldn't actually prevent recurrence.
5. Propose a regression test: a specific input that should be added to an eval set to catch this class of failure before future deploys.
Failure transcript/logs: failure_transcript
System prompt at time of failure: system_prompt