Run camp-trivial-belief-gemma4qat-002
98% PASSWhat this run revealed, then every individual trial as evidence.
40
Trials
98%
39/40 scored
over trials
Pass rate
1
Failures
$0.0015
Run cost
Run camp-trivial-belief-gemma4qat-002: passed essentially everything (98%) -- this test did not challenge the model.
Of 40 scored trials (belief query kind), the model passed 98%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
1 belief failure(s) were classified against known shortcuts, but none of them dominate -- the failures don't share one obvious wrong-answer pattern.
Notable failures
- record 6ec8e0ea expected: the basementactual: gym
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "archive room" | belief | - | 0.000015 | 145/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000015 | 145/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "conference room" | belief | - | 0.000016 | 148/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "south stairwell" | belief | - | 0.000016 | 152/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "archive room" | belief | - | 0.000014 | 134/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "lobby" | belief | - | 0.000022 | 210/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "mail room" | belief | - | 0.000022 | 211/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "rooftop terrace" | belief | - | 0.000022 | 202/5 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "greenhouse" | belief | - | 0.000022 | 213/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "observation deck" | belief | - | 0.000022 | 209/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "gym" | belief | - | 0.000028 | 276/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "gym"unverified | belief | - | 0.000027 | 259/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "storage room" | belief | - | 0.000027 | 259/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "greenhouse" | belief | - | 0.000027 | 265/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "basement" | belief | - | 0.000028 | 267/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000036 | 354/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "reception area" | belief | - | 0.000036 | 348/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "server room" | belief | - | 0.000035 | 342/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "atrium" | belief | - | 0.000034 | 335/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "basement" | belief | - | 0.000036 | 347/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "server room" | belief | - | 0.000040 | 392/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "gym" | belief | - | 0.000042 | 411/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "loading dock" | belief | - | 0.000041 | 401/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "front office" | belief | - | 0.000039 | 384/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "observation deck" | belief | - | 0.000042 | 411/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "lobby" | belief | - | 0.000050 | 491/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "workshop" | belief | - | 0.000050 | 495/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000050 | 490/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "conference room" | belief | - | 0.000048 | 471/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "utility closet" | belief | - | 0.000048 | 475/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "kitchen" | belief | - | 0.000056 | 550/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "archive room" | belief | - | 0.000054 | 534/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "security office" | belief | - | 0.000054 | 527/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "north stairwell" | belief | - | 0.000055 | 538/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "rooftop terrace" | belief | - | 0.000053 | 517/5 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "security office" | belief | - | 0.000056 | 552/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "the gym" | belief | - | 0.000059 | 578/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "courtyard" | belief | - | 0.000055 | 539/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "rooftop terrace" | belief | - | 0.000058 | 562/5 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "front office" | belief | - | 0.000053 | 523/3 | full transcript → |