The quality gate.
Rules you can read.

BPOS's quality gate decides what an agent's work does next. Each acceptance criterion is judged three times by a judge model; work that fails the check declared for its kind of file scores zero; written rules then decide whether the work commits on its own, is sent back by the platform, or goes to a person.

This article belongs to What BPOS runs. The criteria it judges are written before the work, in Every job is a contract.

What a quality gate has to do for agents

In ordinary software, a quality gate is an automatic check between one stage and the next: the build passes, the tests pass, the work moves on. Agents need more, because their work can build, pass and still not be what was asked. So our gate judges two things: whether the work does what its contract says, and whether it survives the checks its kind of file must pass. Then it applies rules that anyone can read.

Step one: judge each criterion three times

A judge model reads the work against each acceptance criterion and says whether it is met, with its evidence. One judgement can flip on noise, so each criterion is judged three times, independently, and the majority decides. The job's score, EQS, is the share of its criteria that passed, from 0 to 1. In the 14 days to 9 October 2026, 1,045 criteria were judged this way and 84.6% passed. A judge model is useful and fallible; LLM-as-a-Judge covers how far it can be trusted.

Step two: the check declared for the kind of file

Work that fails the check declared for its kind of file scores 0, whatever its criteria say: a build for Go, a type check for TypeScript, a validation for a database schema. Which check applies is declared in the platform's graph, not in its code, so a new language is a new declaration. A kind of file with no declared check is judged on its criteria alone.

Step three: the rules

• Tests failed their last run: the commit is blocked. A failed test never commits on its own.
• The three judgements disagree: when their spread passes a limit, the work goes to review instead of committing. The limit is 0.2, and it lives in the platform's graph, not its code. One Lite job went to review at 0.27, because one judge could not see the evidence for two criteria that the other two could.
• The score is below 0.60: the work goes to review.
• The sprint asks for a person on every job: it gets one, whatever the score. In the 30 days to 9 October 2026, 86 of 268 sprints did.
• Otherwise the work commits on its own. At the end of a sprint only approved work is committed, and every commit, pull and push is recorded.

Before a person: the platform's own review

Review used to mean waiting for a person, however mechanical the decision. Since 3 October, work routed to review by its score is first read by the platform's failure judge: the attempt's record, its case file and the parts of the graph the failure names. It answers retry or park, with its grounds and its confidence. When it is confident the work should be retried, the platform sends it back itself, and the sprint continues. Since 11 October, whether to send it back again is decided by the job's attempts: while their errors keep shrinking, it may go on; a limit of eight remains only as a safety net. When to stop an agent loop explains the rule. Everything else goes to a person, found in an order the judge's page declares, with the diagnosis already written. The platform sends back the work's own failures; a person gets the rest.

It is young. Its first verdicts came on 3 and 4 October: 14 in all, of which the platform sent back 6, left 6 for a person, and failed to send back 2. Every verdict is recorded, with who decided and how long the work waited.

Where it falls short

• The score does not see the tests. A failed test blocks the commit, but it does not lower the score. On a small sample in the 14 days to 4 October, executions whose tests failed averaged 0.975 and those whose tests passed averaged 0.980. Those scores feed what the platform learns, so a failure can be learned as a success. Measured, not yet fixed.
• The commit bar is still in our code. The disagreement limit and the number of send-backs moved into the graph in early October; the 0.60 bar has not yet. Until it does, changing it is an engine release rather than a recorded decision.
• Some criteria ask too little. About 1 in 6 only check that a name exists in the code. The gate can only be as good as the contract it judges.

What we learned building it

Most of this gate was shaped by things that did not work. The articles below are the record: a judge that saturated at a perfect score, checklists that changed nothing, a result we published and then had to correct, a green build that proved less than it seemed, and the scaffolding we deleted once it had become a cage.

Common questions

Sources

• Our own records, as of 10 October 2026: criteria judged over 14 days; sprints asking for a person over 30 days; the platform review's verdicts of 3 and 4 October; the test-run sample over the 14 days to 4 October.