Our agents kept telling us the job was doomed. Nothing was listening.

A supervised agent platform is a system where AI workers write code and humans approve it before it ships. Ours ran a night of maintenance on itself on 15–16 July 2026 — its own quality gate, its own prompt assembly — and the result was the most useful failure report we've had all year.

Here's the honest version.

The setup

We pointed the platform at five of its own defects. Plan the work, decompose it into jobs, let the workers edit the code, score the output, put it in front of a human reviewer. The same loop we run on customer projects, aimed inward.

The models did their part. Every job that failed, failed for a platform reason. Five defects came out of one night, and four of them share a single shape: the platform discarding correct work.

The five defects

1. A correct job scored zero because there was nothing left to do

A job finished its edit, the engine restarted mid-run, and the re-run found the work already on disk. It applied zero edits — because zero edits were needed — and the scorer read "zero applied" as total failure. Correct code, score 0.000. The score measured our interruption, not the work.

2. Reviewer feedback silently shrank the job

A reviewer rejected a passing job with precise notes: fix these two things, keep the design. The re-run treated the notes as the whole objective. Everything the notes didn't mention fell out of scope — including a piece the previous pass had built correctly — and got reverted to the pre-job version. The good 90% was undone along with the bad 10%, because the feedback replaced the contract instead of amending it.

3. Pressing "reject" made a passing implementation unfindable

When the re-run broke the build, the retry system restored a "frozen base" — the state from before the job ever ran. A reviewed, compiling, passing implementation vanished from the working tree.

We spent the night believing that work was destroyed. It wasn't. It had been snapshotted at review time, correctly, and labelled with the job it belonged to — and it was sitting on disk the whole time. The platform had done everything right except the last step. The snapshot service returns the label; the code that calls it parses the response into a struct that never declared a label field. So the label was dropped on arrival, the column that exists to hold it stayed empty, and nothing in the system could ask the one question that mattered: what did this job look like when the human approved it?

Not data loss. A missing field. Review — the human gate the whole "supervised" premise rests on — was, mechanically, a one-way door with the key on the other side.

4. A retry that couldn't find a one-line error rewrote everything

The compiler error named its own location — file and line. The retry's targeting came back empty, and its response to "I can't localize this" was to implicate all 15 of 15 methods in the file and budget a full rewrite. A one-line fix, escalated to maximum blast radius, because the fallback fails open instead of closed.

5. The agent called the trap in advance. Three times.

Before failing, the worker wrote in its reasoning trace: "the instructions contradict each other." It was right — the job's constraints and its objective were mutually unsatisfiable, manufactured by a prompt-assembly bug on our side. It said so three times across two passes of the same job: first that the instructions were confusing, then that they contradicted each other, then that the confusion stemmed from conflicting instructions. It was overruled three times, because being overruled was the only option available to it. The trajectory ended at 0.000. All three warnings were sitting in a database table that nothing joins.

The one-line diagnosis

The agent isn't failing to warn us. The protocol is failing to listen — twice. Once at decision time: there is no legal "stop" move, so a worker that sees the trap must still walk into it. Its only permitted outputs are edits. And once at retry time: the rollback discards the very state the warning described.

Count the channels. The reasoning trace. The reviewer's note. The interface contract. The checkpoint label. Every one of them written down correctly, on time, by the platform itself — and every one of them unread.

Our failure mode isn't losing information. It's never reading what we already wrote down.

What we're changing

Five invariants. In progress, not shipped — we'll publish the follow-up when they're live and measured.

• A worker may refuse. "Blocked: these instructions conflict" becomes a legal output that halts the job and surfaces the conflict to review — instead of forcing code that can't be right.
• Review is a ratchet. Work that passed review is checkpointed and reachable. A retry may supersede it, never silently strand it.
• Feedback amends the contract. A rejection note adds constraints on top of what the reviewed pass built; it never replaces the job's scope. What was built and approved stays in scope by construction.
• Failed targeting shrinks, never explodes. If the retry can't localize an error, it scopes to the location the compiler already named. Escalating to "rewrite everything" requires a human.
• Something reads the reasoning. A small, cheap model sits in the loop and reads what workers write at decision time — before budgets burn.

We tried the other direction once: when a job looked hard, hand it to a bigger, more expensive model. We retired that path — it cost more than it returned. This is the same lesson from the other end. The expensive insight was already in the trace, for free. It didn't need a bigger model. It needed a reader.

Why this matters beyond us

If you're training anything on agent telemetry — routing decisions, retry policies, quality predictors — your labels are only as honest as your platform. Ours was recording "failure" on jobs where the model was right and the platform lost the work. Those labels don't average out. They teach a learner that good moves fail.

We've written before that the model is the smallest part of execution quality. This is what that looks like from inside a bad night: five defects, zero of them model failures, and the strongest early-warning signal in the whole system sitting unread in a table — written by a worker, at no extra cost, while it was being forced to do the wrong thing.

One ruined file on a Tuesday is a fair price for finding that out before a month of training on poisoned labels.

Questions

This is an engineering note from Bouletteproof's internal delivery platform. The fixes above are in progress, not shipped; we'll publish the follow-up when they're live and measured. Every number here comes from our execution data. Where we don't have a receipt, we say so. Related: The Checklists That Did Nothing · We Deleted Our Own Quality Checks