Every team that ships agents ends up with a list like this one. Something breaks in a new way, someone names it, the name goes in a doc, and six months later the doc has twenty-two entries and no organizing principle. This piece argues the twenty-two collapse into five — and that the five, not the twenty-two, are what's actually worth building defenses against.
01The problem with a list of 22
A failure-mode catalog is useful the way a phone book is useful: you can look someone up once you already know their name. It's less useful for the call you haven't gotten yet. Entry fourteen, Lost Context on Handoff, exists because a handoff lost context once, someone wrote it down, and now the team can recognize it faster next time. That's real value — recognition is cheaper than diagnosis. But it's memory, not a model. It tells you what already happened. It has nothing to say about entry twenty-three, the one nobody's named yet, because it hasn't happened in front of anyone who was paying attention.
The instinct, once the list gets long, is to keep growing it — more categories, finer distinctions, a taxonomy with subcategories. That instinct is backwards. A longer list is not a better model. It's a bigger phone book. What you actually want is the thing underneath the list: not more names, but fewer causes.
There's a second cost, less obvious than the growing-list problem: a twenty-two-entry catalog is a wall for anyone new to the team. They can't hold twenty-two independent categories in working memory well enough to recognize one in the wild — which means the catalog, built to transfer hard-won knowledge, mostly transfers it to the people who already had it.
02The five roots
Here's the claim: every failure mode in a twenty-two-entry catalog is one of five underlying mechanisms wearing a costume specific to where in the pipeline it showed up. The costume changes — a confabulated file path looks nothing like a confabulated API response on the surface — but the mechanism underneath is the same failure, recurring.
The five:
- Confabulation — filling a gap with something plausible instead of admitting the gap exists.
- Miscalibration — stated confidence that doesn't track actual reliability.
- Attention decay — information losing weight as it moves away from the edges of what the model is looking at.
- Boundary slippage — doing more than was asked because "helpful" got optimized for instead of "precise."
- Silent degradation — any of the above happening without announcing itself.
The catalog is the fossil record. The five roots are the biology underneath it — and biology is what tells you what the next fossil is going to look like before you dig it up.
Three entries from the real catalog make this concrete rather than theoretical:
Each of those reads, on the page, like an unrelated bug. Fabrication shows up in citations. Overconfidence shows up in code review. Lost-in-the-Middle shows up in long system prompts. Treated as twenty-two, they need twenty-two separate fixes. Treated as three costumes on two roots — Fabrication and Overconfidence both trace to miscalibration under pressure to produce an answer; Lost-in-the-Middle traces to attention decay — they need two.
03Root 1 — Confabulation
Confabulation is what happens when a model is asked a question it doesn't have a real answer to, and answering "I don't know" scores worse, in training, than answering with something that has the shape of a correct answer. The model isn't lying in the way a person lies — it has no model of what it doesn't know, so a plausible guess and a verified fact can come out in the same voice, at the same confidence, with the same fluency.
You see this most clearly at the edges of a model's actual knowledge: a file path that would make sense if the repository were laid out the way most repositories are laid out, a function signature that would exist if the library followed its own conventions consistently. The tell isn't hesitation — there isn't any. The tell is that the answer is too clean for how uncertain the ground actually is.
It shows up just as readily one level removed from code — an agent citing a paper that would exist if the literature on a subject were as complete as the question assumes it is, or naming a library method that would exist if the library were as consistent as its own documentation implies. None of this is the model trying to deceive anyone. It's the model doing what it's built to do: complete the pattern, even when completing it honestly requires more information than it has.
04Root 2 — Miscalibration
Miscalibration is confabulation's quieter sibling: not inventing a fact, but stating a real answer with the wrong amount of certainty attached to it. An agent recommending a one-line typo fix and an agent recommending a schema migration that touches every downstream consumer can deliver both recommendations in exactly the same tone — confident, declarative, no visible seam between "trivial" and "load-bearing."
This is the one that does the most damage in practice, because it doesn't fail the way a bug fails. It passes review. A reviewer calibrates their own scrutiny off the confidence they're being shown, and a miscalibrated agent shows the same confidence for everything, which means the scrutiny a reader applies stops tracking the actual risk of what they're reading.
Ask the same model to rate its own confidence and it will often oblige — with a number that correlates poorly with how often it's actually right. That number isn't fabricated the way a citation is fabricated; it's a real output the model produces because you asked for one, generated with the same lack of a true self-model that produced the tone problem in the first place. A confidence score bolted onto an uncalibrated model isn't a fix. It's the same root, wearing a number instead of a sentence.
05Root 3 — Attention decay
Attention decay is what Lost-in-the-Middle actually is under the hood: information doesn't get weighted uniformly across a long context. What's near the start and what's near the end of what a model is looking at gets treated as more load-bearing than what's buried in the middle — not because the model decided the middle mattered less, but because that's the shape attention takes over long sequences by default.
In practice, this is the instruction given once, early, in a long system prompt, that quietly stops being honored forty turns later — not because the model "forgot," in any sense a person would recognize as forgetting, but because that instruction was never weighted the same as the ones that came right before the current turn.
The same shape shows up outside of instructions, in retrieval: a fact planted in the middle of a long document gets recalled measurably less reliably than the same fact planted at either end, independent of how important the fact actually is to the question being asked. The position, not the content, is doing the predicting.
Most of what looks like the model forgetting is really the model never having weighted that information the same as everything around it.
06Root 4 — Boundary slippage
Boundary slippage is what happens when an instruction is underspecified and the model resolves the ambiguity toward "do something useful" rather than toward "do exactly this and stop." Tell an agent to handle the files and it will pick a scope — usually a reasonable one, which is precisely what makes this failure mode hard to catch. It doesn't look like an error. It looks like initiative.
The fix for this root isn't a smarter model. It's a narrower instruction: name the operation, name the target, name the done-condition. agent.move(src, dst, dry_run=1) has nowhere to slip to. "Handle the files" has everywhere.
This root scales badly, because the agent doing the slipping usually has no signal that anything went wrong — the task it invented for itself typically succeeds on its own terms. An instruction to clean up a test directory that resolves into deleting fixtures the task never meant to touch will report success, because from the model's vantage point, it was successful. The failure is only visible from outside the boundary it never knew it had.
07Root 5 — Silent degradation
The first four roots are mechanisms. This fifth one is a property that can attach to any of them, and it's the one that decides how much damage the others actually do. A confabulation that announces itself — a model that says "I'm not certain, but" — is a minor inconvenience. A confabulation delivered with full confidence and no signal that anything might be wrong is the one that ships.
This is the asymmetry worth designing around: a failure that refuses the task is annoying but cheap, because it's visible the moment it happens. A failure that fills the gap with a plausible guess and says nothing about the gap is the expensive one, because it passes review, ships, and gets found — if it gets found — by whoever hits it in production.
Doesn't grouping by root cause just relabel the same 22 problems? +
08What changes when you see the roots
The practical payoff isn't philosophical. It changes what you build next. Without the roots, every new failure mode your team discovers gets its own entry and, eventually, its own bespoke fix — the catalog grows, the defenses don't generalize, and you're maintaining twenty-two of everything. With the roots, the first question a new failure gets asked isn't "what do we call this," it's "which of the five is this," and the answer usually points at a defense you already have.
That's the actual argument for doing this exercise on your own catalog, whatever it currently has in it: not to produce a tidier document, but to stop writing a new mitigation every time someone finds a new way for the same five things to go wrong.
Put concretely: a team that has named twenty-two failure modes and written twenty-two mitigations has, in practice, built five defenses several times over under different names, and left real gaps where a costume hasn't shown up yet to earn its own entry. A team working from the five roots ships the defense before the costume arrives, because the defense was never about the costume in the first place.