Run camp-full_map-gemma4qat-001/breaking_point
32% PASSWhat this run revealed, then every individual trial as evidence.
56
Trials
32%
18/56 scored
over trials
Pass rate
38
Failures
$0.0021
Run cost
Run camp-full_map-gemma4qat-001/breaking_point: 32% pass rate on belief queries.
Of 56 scored trials (belief query kind), the model passed 32% and failed 68%.
38 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
In 19 of 38 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.
Notable failures
- record 0a7085a2 expected: the print roomactual: reception area
- record 5be1c488 expected: the east wingactual: west wing
- record d4139392 expected: the kitchenactual: front office
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "cafeteria" | belief | - | 0.000023 | 220/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "supply closet" | belief | - | 0.000024 | 231/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "lobby" | belief | - | 0.000024 | 238/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "rooftop terrace" | belief | - | 0.000025 | 238/5 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "reception area"unverified | belief | - | 0.000023 | 217/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "photo studio" | belief | - | 0.000025 | 239/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "west wing"unverified | belief | - | 0.000025 | 237/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "front office"unverified | belief | - | 0.000054 | 530/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "boiler room"unverified | belief | - | 0.000053 | 525/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "courtyard" | belief | - | 0.000054 | 534/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000053 | 526/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "security office" | belief | - | 0.000055 | 542/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000054 | 531/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "observation deck"unverified | belief | - | 0.000053 | 519/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "library" | belief | - | 0.000040 | 392/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "boiler room" | belief | - | 0.000039 | 385/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "workshop" | belief | - | 0.000040 | 391/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "cafeteria"unverified | belief | - | 0.000039 | 382/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "greenhouse"unverified | belief | - | 0.000040 | 391/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "server room"unverified | belief | - | 0.000040 | 386/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "workshop" | belief | - | 0.000038 | 373/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "east wing"unverified | belief | - | 0.000048 | 471/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "print room" | belief | - | 0.000049 | 480/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "conference room"unverified | belief | - | 0.000049 | 476/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "security office"unverified | belief | - | 0.000045 | 438/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "photo studio"unverified | belief | - | 0.000049 | 476/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "gym"unverified | belief | - | 0.000047 | 459/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000046 | 453/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "loading dock"unverified | belief | - | 0.000043 | 420/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000041 | 400/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "storage room" | belief | - | 0.000042 | 415/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "utility closet"unverified | belief | - | 0.000043 | 420/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "kitchen"unverified | belief | - | 0.000043 | 419/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "loading dock"unverified | belief | - | 0.000044 | 430/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "print room"unverified | belief | - | 0.000044 | 429/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000044 | 433/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000041 | 405/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "boiler room"unverified | belief | - | 0.000042 | 411/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000043 | 423/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000042 | 410/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "courtyard"unverified | belief | - | 0.000043 | 420/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "greenhouse"unverified | belief | - | 0.000043 | 425/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "storage room" | belief | - | 0.000025 | 244/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "north stairwell" | belief | - | 0.000026 | 249/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "break room" | belief | - | 0.000023 | 219/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "storage room"unverified | belief | - | 0.000024 | 234/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "gym" | belief | - | 0.000024 | 235/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "server room"unverified | belief | - | 0.000024 | 230/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000023 | 225/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "photo studio"unverified | belief | - | 0.000025 | 240/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "supply closet"unverified | belief | - | 0.000023 | 225/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "reception area" | belief | - | 0.000023 | 221/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "reception area"unverified | belief | - | 0.000025 | 235/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000023 | 228/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "unknown"unverified | belief | - | 0.000022 | 218/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "server room"unverified | belief | - | 0.000025 | 238/3 | full transcript → |