Why AI Models and Agents Get Worse the Longer They Run
Dylan Telano · Oct 27, 2026 · 5 min read
TL;DR: AI models and long-horizon AI agents get worse the longer they run. The context fills with old versions of every fact, and each call burns reasoning to work out which one still holds.
Long-horizon AI gets worse with length
You have seen this if you build with AI every day. A Claude Code or Codex session starts sharp. A few hours in, the context holds 100,000 tokens, then 500,000, then 800,000. The model reverts a fix you made an hour ago, calls a function you renamed, or picks up a plan you dropped after lunch. These slips are the clearest tell, and they come more often as the token count climbs.
Agents hit the same wall. A support agent works through a return, an address change, a discount and a refund. A research, legal or sales agent runs for a week across email, tickets and documents. Each step is easy. The facts change between the steps.
Bigger context windows make this worse. Chroma’s research team calls it context rot: across the 18 models they tested, output got less reliable as the input grew. Every old instruction, superseded fact and abandoned plan stays in the window, so the model has more to confuse. Before it can do the task, it has to work out which version of each fact is current, and it pays for that reasoning again on every call. Those tokens go to rebuilding the situation, not to the work, and the model still guesses wrong often enough to hurt.
When it guesses wrong, you pay twice. Someone rolls the work back, a person steps in, or you ask a second model to repair the first. Each repair burns more tokens.
The usual fixes trade one loss for another. Compaction and summaries shrink the window and drop details you need later. A memory layer or a hand-kept notes file, such as a CLAUDE.md or a progress log, carries some facts forward, but someone decides what goes in, and stale entries stay until a person deletes them. Skip all of it and you pay for a huge context that still can’t tell where the work stands.
What models and agents lack is bookkeeping: a record of which facts still hold.
Why it happens: old tickets on the rail
Picture a busy kitchen. Orders arrive as tickets on a rail. A table changes its order, and a new ticket goes up. The old one stays. Now the cook has two tickets for table nine, and both look official.
A model’s context window is that rail. Every instruction, tool result and customer message stays up, in the order it arrived. Nothing gets taken down when it stops being true. Three things go wrong:
- Stale facts. The customer gave a new address. The old one is still in the window, and it was mentioned more often.
- Superseded instructions. The manager raised the discount cap. The original policy is still there, word for word.
- Context drift. As the rail fills, the model’s sense of what matters shifts toward whatever is recent, loud or repeated, which is not the same as what is true.
The model knows how to cook. It can’t tell which ticket is live.
The usual fixes make the rail longer
The instinct is to buy a bigger model or a bigger context window. Both help a little. Neither takes old tickets down.
| Fix | What it does | What it leaves on the rail |
|---|---|---|
| Bigger context window | Fits more tickets | Every stale one, now with more room to hide |
| Bigger model | Reads tickets more carefully | The work of deciding which is live, at a higher price per step |
| Summarizing the history | Shortens the rail | Whatever the summary got wrong, now stated as fact |
| Memory layer | Recalls what was said before | What was said, not what is true now |
The last row is the subtle one. Memory answers “what happened?” Long tasks need “what’s true now?” We wrote about that gap in Memory Isn’t State.
What fixes it: someone who clears the rail
Good kitchens have an expediter. The expediter keeps one rail of live tickets: pull the old order when the new one lands, and tell the cook what changed.
That is what a state layer does for a model or an agent. It sits beside the model and keeps three things current: what happened, what’s true now, and what’s next. When a fact changes, the old version is marked superseded, not just buried. Before the model or agent acts, it checks against the live state, not the whole history.
Our bet is that this matters more than model size on long tasks, and that it lets a cheaper model do work that would otherwise need an expensive one.
What this means for you
If you use AI for more than a few minutes at a time, you have this problem, and you have probably never seen it named.
- You keep one long chat going in ChatGPT and it starts mixing up what you decided an hour ago.
- You run an agent system like Grok Bot, Muse or Dots, and it acts on a plan you already changed.
- You build every day in Claude Code or another coding agent, and the session drifts as the context grows.
- You run a customer-facing agent for support, legal work, auto insurance claims or anything else, and it applies a rule, price or address that is no longer current.
In every case you pay more for worse results. Your AI spends tokens rebuilding the situation on every call, still gets it wrong, and someone pays again to fix it. The cause is the same each time: the system reasons and remembers, but nothing in it keeps track of what is true now.
Reasoning and memory are two of the three things AI needs. State is the third. Add it, and the same models finish more of the work for less money, without someone cleaning up after them. Our bet is that the gain compounds: the longer your AI runs, the more a current picture of the facts is worth.
Give your AI state, and stop paying it to rediscover where it is.