What BPOS runs · Learn

A pass is not knowledge.
So we ask why it worked.

Philosophy settled in 1963 that a true, justified belief can still be luck. AI agents produce true, justified results all day. This is how BPOS tells the skill from the luck.

The Gettier problem is the gap between a belief that is true and justified, and knowledge: the belief can be right by luck. For AI agents, a passing job is exactly that kind of belief. BPOS's epistemic round closes the gap by asking, when a job ends, why it ended where it did, successes included, and keeping only answers that verify something.

This article belongs to Procedural memory for AI agents: the round is how the record of what happened becomes a lesson for the procedure.

A pass is not knowledge, so we ask why it worked. Every job ends: a pass, a failure, a rejection. The epistemic round asks why, in a separate run on its own pages, successes included. It answers five questions: claim, source, method, confidence and verification. An answer is kept only if it verified something, and its confidence is the share of its sources that check out. It names which page should learn, and a separate reading decides from what happened when the change was tried. A success with no reason is luck, and luck does not repeat. 3,428 claims in 30 days, on 279 jobs.

What the Gettier problem is

For a long time, knowledge was defined as justified true belief: you know something if you believe it, it is true, and you have good reason to believe it. In 1963, Edmund Gettier published a three-page paper, Is Justified True Belief Knowledge?, with cases where all three hold and the person still does not know, because the belief is true by luck.

The clearest version came later, from Alvin Goldman in 1976. Henry drives through a county full of fake barn facades, built to look real from the road. He looks at the one real barn and thinks: that is a barn. He is right, his eyes are working, and his reason is good. But he would have thought the same of any facade. His belief is true by luck, and he does not know.

Why it matters for AI agents

A job that passed is Henry at the barn. The score says the work is right, and it is right, and the score came from a reasonable process. But the pass may still be luck. One in six of our acceptance criteria only checks that a name exists in the code. Our score does not see the tests. A criterion can pass for a reason that has nothing to do with the work. A Green Build Is Not Knowledge made this point about pipelines; it holds for every kind of pass.

A system that learns only from passes and failures learns the luck along with the skill. And luck does not repeat. So the question that matters is not only why a job failed. It is why a job that passed, passed.

The epistemic round

When a job ends, BPOS triggers a separate run of its mesh, on a skillset of its own: fourteen pages, never chosen by the job's subject, because a question about why a coding job worked must not be answered by the coding procedure. It runs in the background. When work goes to review, the platform's review waits for the round, so its diagnosis can read the answer.

It asks one question of the job's record: why did this job end where it did? Passes are asked, failures are asked, and since September so are rejections. When a person rejects work, the rejection used to reach the next attempt as instructions and nothing more. Now the round has to explain it.

It reasons the way inquiry has been described since Charles Sanders Peirce: guess the best explanation (abduction), work out what must also be true if it holds (deduction), and check whether the record bears it out (induction). Each step is a page, and so are its disciplines: an absence of evidence is not a refutation; a declared fact is not an inferred one; an instrument must be verified before it is believed.

Five questions, and a gate

Every answer is written as a claim with five parts: the claim itself, its source, the method that established it, the confidence it deserves, and how it will be verified. Those five are the columns of our knowledge graph. The fifth is real: the things a claim cites are marked to be checked again, so a claim cannot outlive the facts it rests on unnoticed.

Then the gate. Confident prose with nothing behind it is worse than no claim, so an answer that verified none of its sources is not written at all. An answer that verified some is written, with a confidence equal to the share of its sources that checked out, and the ones that did not are kept on the claim to be resolved. We used to drop any answer with a single unresolved source. In one day that discarded seven answers out of eight, each over one source among six to twenty-two that did check out. A gate that throws away most of the truth to avoid a little doubt is the wrong gate.

From an answer to a lesson

A claim that explains a job is knowledge about that job. To become a lesson, it has to change the procedure. So the round also says which page should learn from it, and proposes the change. It does not make the change. A separate reading decides whether a proposal becomes a rule on its page, from the proposal, what the round cited, and the recorded outcome of every attempt that was given the proposal to try. Once a rule lands, its weight belongs to what happens next, never to the reading that landed it.

The round learns too. Its own path through its pages is reweighted by its own outcome: the share of its answers that passed the gate. That is a count, not a model's opinion of itself.

What it has done, and where it falls short

Since 3 September the round has written 4,238 claims; in the 30 days to 10 October, 3,428 of them, about 279 jobs. In the same 30 days, 646 proposals to change a page were made across the platform, and 23 became rules.

  • Its confidence is suspiciously high. The average is 0.99: nearly every answer verifies all its sources. That may mean the round is careful, or that the sources it chooses are easy to verify. We have not measured which, and a judge that agrees with itself 99% of the time deserves the same suspicion we gave our LLM judge.
  • We have not reconciled its coverage. It is triggered when a job ends, but we have not yet matched the jobs it explained against every job that ran.
  • Whether a landed rule helped is still the open part. Proposals land from evidence; measuring that the procedure is better afterwards is the part of the loop we are still building.

Common questions

What is the Gettier problem?

In 1963 Edmund Gettier showed that a belief can be true and justified and still not be knowledge, because it is true by luck. It broke the old definition of knowledge as justified true belief.

Why does the Gettier problem matter for AI agents?

Because a passing score is a justified, true belief that the work is right, and it can still be right by luck: a weak criterion, a test that did not look at the thing that mattered. A system that learns only from passes and failures learns the luck along with the skill.

What is BPOS's epistemic round?

A separate run, triggered when a job ends, that asks why the job ended where it did, successes included. It answers five questions, claim, source, method, confidence and verification; keeps an answer only if it verified something; and names the page that should learn from it.

Sources

  • Edmund L. Gettier, "Is Justified True Belief Knowledge?", Analysis 23(6), 1963.
  • Alvin I. Goldman, "Discrimination and Perceptual Knowledge", The Journal of Philosophy 73(20), 1976.
  • Our own records: epistemic claims since 3 September and over the 30 days to 10 October 2026; page proposals and rules landed over the same 30 days.

Related writing