Run camp-ef-belief-gemma4qat-001/C
90% PASSWhat this run revealed, then every individual trial as evidence.
30
Trials
90%
27/30 scored
over trials
Pass rate
3
Failures
$0.0016
Run cost
Run camp-ef-belief-gemma4qat-001/C: 90% pass rate on belief queries.
Of 30 scored trials (belief query kind), the model passed 90% and failed 10%.
3 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Diffuse weakness (below 90% but no outright flip) on: coreference (pass rate 89% at the high end), evidence_span (pass rate 80% at the high end).
In 3 of 3 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.
Notable failures
- record 88e752b6 expected: the kitchenactual: south stairwell
- record d067ef31 expected: the basementactual: library
- record c7aa9710 expected: the gymactual: west wing
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000051 | 503/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "north stairwell" | belief | - | 0.000058 | 567/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "kitchen" | belief | - | 0.000054 | 536/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "storage room" | belief | - | 0.000060 | 586/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "boiler room" | belief | - | 0.000054 | 529/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "archive room" | belief | - | 0.000058 | 569/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "workshop" | belief | - | 0.000058 | 572/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "workshop" | belief | - | 0.000055 | 543/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "cafeteria" | belief | - | 0.000053 | 514/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "photo studio" | belief | - | 0.000053 | 519/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "south stairwell"unverified | belief | - | 0.000052 | 505/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "print room" | belief | - | 0.000049 | 484/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "library"unverified | belief | - | 0.000053 | 522/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "boiler room" | belief | - | 0.000053 | 520/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "archive room" | belief | - | 0.000060 | 592/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "west wing"unverified | belief | - | 0.000050 | 490/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "gym" | belief | - | 0.000061 | 599/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "front office" | belief | - | 0.000058 | 567/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "south stairwell" | belief | - | 0.000054 | 529/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000046 | 450/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "conference room" | belief | - | 0.000058 | 573/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "print room" | belief | - | 0.000053 | 522/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000054 | 526/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "cafeteria" | belief | - | 0.000048 | 466/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "observation deck" | belief | - | 0.000054 | 529/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "photo studio" | belief | - | 0.000048 | 470/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000053 | 524/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "greenhouse" | belief | - | 0.000056 | 550/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "garage" | belief | - | 0.000055 | 547/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "security office" | belief | - | 0.000055 | 538/3 | full transcript → |