From 47 minutes to 6 seconds: the same failure, before and after we learned to listen

A supervised agent platform is a system where AI workers write code and humans approve it before it ships. Last week ours hit the same impossible task twice in one day — once in the morning, once in the evening — and the two outcomes are the clearest before/after we've ever measured on our own infrastructure.

Same file. Same job. Same contradiction. Only the platform changed in between.

The contradiction

The task asked a worker to add a field to a data structure. The worker's permitted editing region was a single function — the structure was outside it. Objective and constraints were mutually unsatisfiable, manufactured by a bug in how we assemble instructions. Not the model's fault, and — this is the part that stung — the model knew. Before producing anything, it wrote into its reasoning trace, in plain English: "the instructions contradict each other."

Nothing read that sentence.

Morning: 47 minutes

With no legal way to refuse, the worker did the only thing the protocol allowed: it produced code it knew couldn't be right. The build broke. Then the machinery made it worse, step by step:

• The retry couldn't localize the error — even though the compiler had named the exact file and line — so it fell back to implicating all 15 functions in the file and budgeted 47 minutes for a full rewrite of what was a one-line fix.
• A rejection from a human reviewer re-ran the job, and the re-run's failure triggered a rollback — to a state from before the job ever ran. A reviewed, working, approved implementation was silently erased.
• The reasoning trace that had called the trap in advance appeared a second time that day, on a different job, with the same sentence. Both jobs scored zero. Both warnings went unread.

Total damage: three re-runs, a destroyed implementation, and a scoring system that recorded all of it as "the model failed."

One more admission, because it's the most useful data point in this story: partway through, a human reviewer — with the full context in front of them — diagnosed the failure wrong. The worker's one-line trace was more accurate than the expensive human reading over its shoulder. The diagnosis was never missing. It was sitting in a table, unread.

Evening: 6 seconds

By evening we had shipped five changes. Same job, re-run:

Worker declares:  the objective requires changing the structure,
                  but my region is limited to one function
Platform:         reviewed work snapshot preserved — nothing reverted
Retry ladder:     ended on pass 0 — a declared contradiction
                  cannot be refined away
Outcome:          routed to human review, conflict stated
                  in the worker's own words

Six seconds. One pass. Zero budget burned walking into the trap. The work a human had already approved stayed on disk. And the deliverable handed to the reviewer wasn't a zero — it was the contradiction itself, precisely stated, ready to be fixed at its source.

How: five changes, one principle

1. A worker may refuse. "These instructions conflict" is now a structured, legal output that halts the run and surfaces the conflict — instead of forcing code that can't be right.
2. Review is a ratchet. Work that passed human review is checkpointed; a retry may supersede it, never silently revert it. Rejecting a job can no longer destroy it.
3. Failed targeting shrinks, never explodes. A retry that can't localize an error stops, instead of rewriting everything. The 47-minute blast radius is structurally impossible now.
4. A reader sits in the loop. A small, cheap model reads what workers write at decision time — the traces, the compiler output, the failure — and adjudicates before retries burn budget. On its first live case it diagnosed a corrupted file from the traces, cited the compiler verbatim as evidence, and the guided retry converged on the next pass.
5. A worker may ask. One question per task, answered by the platform from context the worker lacks, one re-ask. Never a conversation — a single structured exchange, capped deliberately, because an agent that can interrogate its platform freely is a security surface, not a feature.

The principle under all five: the diagnosis was always already there. In the reasoning traces, in the compiler output, in a reviewer's notes, in a checkpoint label. Six separate times in one day, the information existed, deposited, and nothing read it. We didn't make the system smarter. We made it listen — and gave the thing being listened to a legal way to act on what it knows.

Why this matters beyond us

The models were right one hundred percent of the time — in the morning and the evening. The 47-minutes-to-6-seconds difference is entirely platform. If you're evaluating agent systems by swapping models while keeping the scaffolding fixed, you're measuring the small variable. We've written before that the model is the smallest part; this is what the biggest part looks like when you fix it: a 470x reduction in time-to-truth on an identical failure, with the same models on both sides of the line.

And if you're training anything on agent telemetry: every morning-shaped failure in your logs is a poisoned label — a platform defect recorded as a model failure. Ours accumulated for weeks before we noticed. The honest dataset started the day the listening did.

---

Engineering note from Bouletteproof's internal delivery platform (integrated with our context stewardship tooling — read more about Context Steward). The refusal path, the ratchet, and the reader are live and were observed working on real jobs; two of the five changes haven't yet met their trigger conditions in production, and the reader's decision thresholds ship with their measurement debt named — we'll publish the follow-up numbers when the randomized comparison has enough runs to mean something.