A wrong theory in the feedback
makes the next run worse.

Rejection feedback is an instruction, not a comment. When a human rejects an AI-written change and explains why, the explanation is not context for the model. It is the strongest instruction in the next prompt. If the explanation is wrong, the model will faithfully implement the wrong explanation.

The incident

In July 2026, on one of our own sprints, a four-job change touched our acceptance scorer, the component that decides whether AI-written output does what was asked. One job came back for review. The reviewer rejected it with a note that contained a theory about the cause of the problem: reasonable, fluent, and unverified.

The next attempt did not converge toward the fix. It converged toward the theory. The model treated the reviewer's causal story as ground truth, restructured the change around it, and the result was worse than the rejected version. The incident is logged on the sprint record, retry by retry.

This is one incident. We are stating that plainly rather than dressing it as a study: n equals one, on our own platform, fully logged. We publish it because the mechanism is visible in the logs and the fix is checkable by anyone running agent loops.

It is also a different defect from one we have written about before. In an earlier night of maintenance, feedback silently replaced a job's scope: everything the notes did not mention fell out of the contract. This one is worse in a quieter way. The scope survived; the cause was wrong. Feedback that replaces the contract loses work. Feedback that implants a false cause steers the work.

Why it happens

A human reviewer reading a human's code offers a theory as a starting point for discussion. The human on the other end pushes back, tests it, discards it if it fails. A model in an execution loop does none of that. Feedback text arrives with maximal authority: it came from the entity that can accept or reject the work. The model's cheapest path to acceptance is to make the feedback true.

So a wrong theory is not neutral noise. It is negative training data injected at the highest-trust point in the loop.

The rule we adopted

Rejection notes carry only sourced facts: compiler output, log lines, query results, the failing check and what it printed. Nothing else.

Theories are allowed, encouraged even, but they live in discussion, labelled as theories, and they never enter the feedback channel until they have been checked against the evidence. The channel that steers the next attempt is reserved for things that are true and traceable.

The rule took. The rejection notes on our sprint records now open with the verified location and the observed behaviour: the file, the lines, what the code actually does there, checked in the workspace before the note is written. As of this week the platform holds 331 human-review rejection notes written to that standard. The failure mode did not need better models to disappear. It needed a cleaner channel.

What to take from an n of one

Not a benchmark. A checklist item. If you run any loop where humans reject model output with written reasons, audit the reasons: every sentence in a rejection is either a sourced observation or a hypothesis. If it is a hypothesis, it does not belong in the channel yet. The cost of the discipline is one extra verification step for the reviewer. The cost of skipping it is a model that gets worse while doing exactly what it was told.

Common questions