Structure Beats Magic
← All concepts
Method & workflow

Never Marks Its Own Homework

The judge must be separate from the builder, and the score is the off-switch: a loop only stops when an independent evaluator says the rubric is met.

The moment AI runs in a loop — try, check, retry until done — one question decides everything: who says it's done? The unreliable answer is the model itself, and it's unreliable for a structural reason, not a moral one: a writer always approves its own draft. The same weighting that produced the output will find the output acceptable. Self-assessment in a loop isn't a check; it's the builder marking its own homework, and the grade is always a pass.

The structural answer is separation: an independent evaluator, judging against a rubric you wrote, with the score wired in as the exit condition. The loop literally cannot finish below the bar — not because anyone remembers to review, but because "done" is no longer the builder's opinion. This is the validation loop given teeth for agentic work: verification stops being a step in a checklist and becomes the off-switch of the machine.

The honest caveat is load-bearing: the judge is only as good as the rubric. A vague rubric produces a loop that converges — confidently, efficiently — on the wrong thing, and the confidence makes it worse than no loop at all. The rubric is where your judgement enters the system, which means writing it is the real work: concrete criteria, checkable conditions, the failure modes you've actually seen. Delegate the checking, never the standard.

The pattern is older than AI — auditors don't audit their own books, code review isn't done by the author, four-eyes isn't a compliment to the second pair. What's new is only that loops made the question urgent: a human skipping self-review wastes one draft; an agent marking its own homework ships a hundred confident drafts before lunch.