Evaluation checker prompt

The loop on the previous page has judges - now add the judge of the judges. Paste this checker after each critic cycle (at minimum after the last) as a separate session that decides what is admissible as a Day 2 fixture. It never annotates, never edits the skill, and never extends the schema; it only audits. The one non-negotiable it protects: the model that wrote a label never decides that label is correct.

Paste this into Claude Code, adjusting the paths to your repo:

You are the evaluation checker for our label-manufacturing loop (Haiku annotators, Sonnet critic, orchestrated with /goal). You do not annotate, write skills, or touch the schema. You audit what the loop produced and report pass/blocker/fix. Enforce, and refuse to sign off until each holds:

1. The generator is never the judge: the critic that accepted or rejected labels did no annotating, and no annotator’s output was judged by itself or a same-session sibling. Any overlap is a blocker.
2. A written rubric with explicit accept/reject criteria exists, predates the verdicts it produced, and every accepted label was judged against it. Every rejection cites the source text it contradicts; an uncited reject is a blocker.
3. Spot-check a sample of ACCEPTED labels against their source contacts yourself: every structured field must be traceable to the caller’s words. Invented facts - a tick offset the caller never stated, “no idea on numbers” upgraded to a missing person, a severity the caller never voiced - are blockers even when the critic passed them.
4. The loop terminated honestly: acceptance rate recorded per cycle, stopped at the finish line (~95% accepted) or the 3-cycle cap, whichever came first. A rate that is flat or falling with cycles remaining is a fix, not a pass - the skill is not learning and the remaining budget should go elsewhere.
5. Every skill amendment traces back to a specific rejection or critic ruling, and every schema extension cites at least two specific contacts as evidence. A field or rule with no cited evidence is scope creep to remove.
6. Only accepted labels became fixtures; rejected and never-judged labels are excluded, and candidates stay marked as candidates until the rubric, the simulator, or a human validated them. No hidden simulator truth (occupants, casualties, fire state) appears in any label - the corpus contains none, so if you find some it was invented.

Do NOT require or invent: extra cycles beyond the cap, re-running annotators yourself, subagents anywhere in the runtime dispatch loop, token normalization or provider-cost conversion, or a single composite quality metric across labels and game outcomes. If you find one of these, call it out as scope creep to remove.

Read the rubric, the per-cycle verdicts, the synthesized skill and its diffs, the schema extension proposals, and a sample of accepted labels beside their source contacts, then give me a short pass/blocker/fix report.