Evidence
Agent code quality decays under iteration, and the cheapest fix for it has been tested and does not work. Both of those are measured. This page is the measurement, followed by one record of the same standard applied to us.
What the benchmark measures
Section titled “What the benchmark measures”SlopCodeBench (Orlanski et al., Mar 2026) chains an agent across 93 checkpoints of extending its own prior code. It is the closest published proxy for maintenance work: the agent inherits its own decisions and has to keep building on them.
| Element | Setup |
|---|---|
| Task shape | Extend the agent’s own prior code, one checkpoint at a time |
| Checkpoints | 93 |
| Models evaluated | 11 |
| Specification | Fixes external behavior only |
| Tests | Hidden from the agent |
| Context carried between checkpoints | None |
| Human comparison panel | 48 maintained open-source repositories |
What it found
Section titled “What it found”| Measure | Result |
|---|---|
| Problems solved end to end | None, across all 11 models |
| Best strict pass rate | 17.2%, falling to 0.5% by the final checkpoint |
| Trajectories with degrading code quality | 80 to 90% |
| Cost per checkpoint | 2.9x growth, with no gain in correctness |
| Verbosity against the human panel | 2.2x, widening every iteration |
| Human panel across the same span | Flat |
The cost line is the one that reframes the problem. Spending more per checkpoint buys more output and does not buy more correctness, so the decay is not a budget question.
The cheap fix, tested
Section titled “The cheap fix, tested”Telling the model to write better code is the obvious first answer, and it is the one the paper closes off.
| Quality-aware prompting | Effect |
|---|---|
| Initial verbosity | Down by about a third |
| Rate of decay | Unchanged |
| Pass-rate subtypes | Statistically unchanged |
| Cost | Up to 48% higher |
Better instructions improve the first checkpoint and do not survive iteration.
What the paper shows, and what we claim
Section titled “What the paper shows, and what we claim”The authors name one intervention they did not test: enforcing structural discipline across checkpoints, through training or through tooling. Flow-Next is the tooling half of that sentence. Gates the agent cannot skip. A different model reviewing every plan and every implementation. A receipt keyed to the exact commit behind every claim of done. Specs, decisions, and evidence as files in the repository, so the next checkpoint inherits a record instead of a scrollback.
The paper does not test Flow-Next, and nothing here claims it does. It establishes two things: the decay is real and measured, and the cheapest alternative to a verification system does not work. The verification spine describes what we built on top of that.
The plan-sync skip gate we removed
Section titled “The plan-sync skip gate we removed”/flow-next:work spawns a plan-sync agent after every completed task, to check whether the implementation invalidated a downstream plan. Each spawn costs 70 to 90 thousand tokens. We designed, built, and fully tested a deterministic gate that would skip the spawn when a task provably could not have affected anything downstream.
Cross-repo validation replayed 27 completed-task scenarios from three external repositories against ground truth from the real plan-sync agent. The gate produced a false skip: a change that altered a downstream task’s meaning while touching no file and no symbol that task referenced. Its skip rate was 1 in 15, against a design threshold of 50 percent.
The gate was removed from the shipped CLI. The decision record says not to try again.