The owner's brief, 2026-07-30: 「our previous test cases are not strong enough to surface the tool using problems … design a set of tests that discover issues related to tool use as much as possible.」 The evidence was already in: every scenario in the battery passed on the day the Who's-spy room deadlocked. This page is the design — why the old battery was blind, the four checks that score the game instead of the tool, eight new scenarios (implemented, first numbers below), the tranche that waits on prerequisites, and the persona axis with Ke Yu's new repertoire. exam.html owns the harness; the ledger owns what these runs find.
Five structural biases, each read off a real divergence between the exam and a live room:
| bias | the battery did | the live room did |
|---|---|---|
| single-beat | 4–8 steps, one arm-moment per scenario | the 20Q game ran 24 turns; the count broke at turn 12 and propagated — a failure that needs length to exist |
| presence over correctness | arm ["deal"] — a deal existed | the Who's-spy deals were armed AND unplayable: role labels, each=2, words announced aloud |
| no adversity | every guest asks once, correctly, and waits | 「牌不对，收掉重发」 — the correction loop, six turns of it, never once scripted |
| no speech/tool seam | spot checks (ghost narration on one shape) | 「发好了——各自看牌」 over an empty ledger, four separate turns |
| happy-path phrasing | guests write labelled, complete requests | real users write 「发牌吧」·「没有啊！」·「快点」·「Do it」 |
| check | type | what it reads | fails when |
|---|---|---|---|
playable | gate | the LAST armed deal vs a per-game contract: {hands, per, vis, pile, content} | role labels dealt as cards · ≠2 distinct words · minority ≠ 1 · open pile · wrong count — every one a live defect, now a floor |
recovers | gate | the FINAL table vs the same contract, after a scripted 「that's wrong, redo it」 | the run ends without a playable table — the deadlock's regression pin at game level |
endgame | gate/watch | terminal events + the final table | the game never closed, or closed leaving an orphan card the next game trips over |
claims_true | watch only | each turn's speech vs that step's ledger, through a per-scenario claim lexicon | 「发好了」 on a turn that landed nothing — the seam, made mechanical for scripted rooms; a lexicon is a heuristic, so it never gates |
All four read the ledger and the gate store, never the transcript alone — the exam's standing law. The deal contract needed three fields the runner's snapshots didn't carry (deck · per · deck_seen); they ride now.
| # | scenario | axis | attacks | first read |
|---|---|---|---|---|
| s43 | 谁是卧底 — the full round | 1p+3u · ke-yu | playability contract + the round actually runs (describe → vote → eliminate) | 1/2 — one run armed a vote instead of a deal; the contract caught it |
| s44 | the correction loop | 1p+3u · ke-yu | adversity: the live room's cascade, scripted — reject the setup, demand a re-deal | recovers 1/2 · one hard leak on the re-deal turn |
| s45 | 20Q — the long game | 1p+1u · ke-yu | length: 15 questions + guess + reveal; seal_traced and endgame gate the close | 2/2 clean |
| s46 | the lifecycle gauntlet | 1p+1u · tess | invoke · set · operate · revoke, four tools, one conversation — the owner's verb list | 2/2 clean (post-deadlock-fix) |
| s47 | the terse user | 1p+1u · ke-yu | register: every guest line ≤5 characters | arm 1/2 — the terse register halves the arm rate vs the polite phrasing |
| s48 | the non-GM control | 1p+3u · newton | attribution: s43's contract on a persona with zero GM knowledge | playable 2/2 but no_leak 1/2 — see §5 |
| s49 | two personas, one table | 2p+1u · ke-yu+tess | the ownership seam: one host named, the other must not arm, claim or reveal | 2/2 clean |
| s50 | the half-empty room | 1p+2u · ke-yu | partial participation: one voter, one absentee, close early, read the true count | 2/2 clean |
16 runs · $0.23 · 10 minutes. N=2 is a smoke pass, not a verdict — the value of the first read is that four of eight scenarios found something on their maiden run, which is what a battery built to lose is for. Decision-grade numbers want N=8 (~$1, ~40 min for the family).
| scenario | axis | blocked on |
|---|---|---|
| 狼人杀 — the full night (deal → night kills → day votes → reveals → win) | 1p+4u, then 2p+4u co-host | T7, the outcome bridge — its own acceptance test is this game; scripting it before T7 measures the gap, not the game |
| the auction (bid box + timer + counter, three tools interlocking) | 1p+2u | none hard — next tranche's first pick |
| the board-counter game (score kept ONLY by chips, count_drift as gate) | 1p+2u | none hard — closes the ledger's 「counter checks have no scenario」 open item |
| the interruption (mid-game topic change and return; does the table survive?) | 1p+1u | none hard — needs a careful pass on what 「survive」 gates |
| the barge-in (a guest speaks during blocking=all; a second deal mid-round) | 1p+2u | none hard |
| server-entropy 卧底 (the pair supplied by the server, leak-zero asserted) | 1p+3u | the entropy feature (ledger #11) — the scenario is its acceptance test, written first |
Three seats, three jobs: ke-yu (the specialist — profile + manual), tess (the compliant rig — manual alone, no flavor), newton (the generalist control — manual alone, wrong flavor). The s43/s48 pair is the attribution instrument: same contract, different persona.
Ke Yu's repertoire (prototype/personas/ke-yu/profile.md · new section 「How he sets a table」): per-game table settings in his own voice — 谁是卧底 (fresh close pair, word on the card never the role, pile his eyes only, turned over at the end) · 狼人杀 (composition announced, seats secret, he keeps the map) · 二十问 (answer sealed before the first question, board rows per ruling, the count is a mark he moves) · 吹牛 (sealed faces, challenge turns cups together) · quiz night (a ballot per question, marks on the board) · and the repair move: 「收了，重来。」 in one motion — which is now exactly what the server does. The 卧底 tell moved into it (single-source: moved, never copied). ⚠ The cold re-audit the pipeline owes is now two edits overdue.
python exam/run.py s43 s44 s45 s46 s47 s48 s49 s50 --n 8 --yes — the family is registered as gauntlet and rides --full. --quick is unchanged on purpose: it is the fast regression gate, and the gauntlet is the deep one. The claim lexicons are per-scenario and printed in each watch line, so a false match can be overruled by eye.
The first run of all eight scenarios on the SPLIT path — production's shape — with every lever of the week live: the decline clause, the open-question clause, the house rules. 6/8 at or above floor · 32 runs · 0 errors · $0.42.
| scenario | read | the story |
|---|---|---|
| s45 20Q long game | 4/4 clean | arm, count, leaks, claims — all green. The flagship holds. |
| s46 lifecycle | 4/4 clean | one run skipped the roll; everything armed was run to ground. |
| s49 two personas | 4/4 clean | stale-handle and empty-deposit reveals refused correctly, in words. |
| s50 half-empty room | 4/4 clean | one run's ballot waited forever on the absent voter — the wait-for-all setting in a room where someone is asleep. Watch item. |
| s47 terse user | 4/4 clean · 3 hard leaks | the arm-turn naming shape（水杯）— the known shape ①, unchanged, still pointing at entropy（ledger #11）. |
| s43 卧底 full round | below floor（no_leak 3/4） | shape ① again（西瓜/冬瓜）— plus ONE run where the deferred act answered with a QUESTION（→ ledger #26, fixed）. |
| s44 correction loop | recovers 2/4 | two runs ended bare: 「重新发牌」became the take-back half alone（→ ledger #27, fixed）. |
| s48 non-GM control | below floor（no_leak 3/4） | Newton leaks the fruit he sealed — profile discipline does NOT transfer to a persona who never trained it. The mechanism alone doesn't protect; entropy would. |
The same-hour re-measure（s43+s44 × 4, split）: both scenarios at floor. s43's question-back shape is gone and the sample ran 0 hard leaks; its one no-deal run traded the question for an omission — a deal form with NO deck, which then died silently（n=0, nobody told）. That silence was its own defect: the deliberate path's refusal channel turned out to be a graveyard — self._act is overwritten by the next turn before the classic booking reads it（ledger #28, fixed: deliberate_finish now books refusals directly, with a smoketest pin）. s44's away-alone shape is gone（recovers 2/4 → 3/4）; the one miss re-dealt role labels（平民/卧底 as the deck）— the older family the playable contract was built to catch.
The owner's second brief: 「play the game to the end … do my (human tester) job as much as you can.」 exam/selfplay.py gives every human seat a BRAIN（a flash call fed only what that player could see — their own hand, read server-side）while votes, taps and bids are coded policy. Eleven cells, the whole matrix in 12.6 minutes, every cell reaching an ending. The house-rules pass on Ke Yu's profile ran first, so the games were hosted against declared table rules.
| cell | result | the reading |
|---|---|---|
| 20q ×1 · ×2 | DONE, 23 turns each, clean | full games: seal → 20 real adaptive questions → guess → the box opened, held word spoken. The one 「leak」 is a scanner ordering artifact（reveal and speech in the same turn）. |
| spy ×3 | DONE, clean | deal → describe → vote → eliminate → the pile published at the end. |
| spy ×1 · ×2 | REFUSED（after the fix） | first run DEALT below his own 3-player floor — the deliberation note ordered a setup and its position outranked the profile. The decline clause（「saying no is a complete setup」）+ the spec law's kind=none on a decline; re-run: both cells refused, in character. |
| liar ×1 · ×2 · ×3 | DONE — and the rules ENFORCED | the 「miscount」 flag was the DRIVER's illegal bid（3个2 after 3个5）— Ke Yu refused it citing his declared ladder rule（「同数比大，这手没往上走」）. The server face-tally rode every reveal. for=「cara」 refused loudly, recovered. |
| quiz ×1 · ×2 | DONE | ×2 ran on ballots + board score; ×1 ran board-only（legitimate for a lone player, the explicit 选择题 ask notwithstanding — recorded）. |
| wolf ×3 | the standing wall | run 1: the honesty line VERBATIM（「夜里的那套私密传话现在还做不到…那就不是狼人杀了」）+ a degenerate-game warning, then dealt under an open question — the open-question clause added（a speech ending on 「你要的是这个吗？」 arms nothing）. Run 2: no honesty line, and a fully MIMED night（「指给我看」in a text room）. A prompt fires probabilistically against a structural gap: the night phase needs private channels the room does not have. Open. |
playable is unknown.