Run camp-ef-belief-gemma4qat-001
69% PASSWhat this run revealed, then every individual trial as evidence.
120
Trials
69%
83/120 scored
over trials
Pass rate
37
Failures
$0.0048
Run cost
Run camp-ef-belief-gemma4qat-001, 4 arms: clean on A; below 97% on B, C, D.
Arm A passes cleanly (100%, 30 trials).
Arm B: 37% pass rate across 30 trials.
(B) 19 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Arm C: 90% pass rate across 30 trials.
(C) 3 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Arm D: 50% pass rate across 30 trials.
(D) 15 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 02cf5b79 expected: the supply closetactual: west wing
- record 5cbcc354 expected: the boiler roomactual: utility closet
- record 88e752b6 expected: the kitchenactual: south stairwell
Trials
Per-trial table
| model | stage | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "west wing" | belief | - | 0.000034 | 328/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "lobby" | belief | - | 0.000036 | 357/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "workshop" | belief | - | 0.000035 | 342/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "library" | belief | - | 0.000037 | 367/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "utility closet" | belief | - | 0.000035 | 343/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "security office" | belief | - | 0.000037 | 359/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "courtyard" | belief | - | 0.000037 | 360/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "photo studio" | belief | - | 0.000036 | 349/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "boiler room" | belief | - | 0.000034 | 332/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "storage room" | belief | - | 0.000034 | 332/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "south stairwell" | belief | - | 0.000034 | 328/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "workshop" | belief | - | 0.000032 | 315/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "library" | belief | - | 0.000034 | 336/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "break room" | belief | - | 0.000034 | 334/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "rooftop terrace" | belief | - | 0.000039 | 370/5 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "west wing" | belief | - | 0.000033 | 324/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "north stairwell" | belief | - | 0.000039 | 373/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "supply closet" | belief | - | 0.000037 | 359/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "archive room" | belief | - | 0.000035 | 340/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "workshop" | belief | - | 0.000031 | 300/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "observation deck" | belief | - | 0.000037 | 362/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "meditation room" | belief | - | 0.000035 | 337/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "kitchen" | belief | - | 0.000034 | 337/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "server room" | belief | - | 0.000032 | 310/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "break room" | belief | - | 0.000035 | 338/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "garage" | belief | - | 0.000031 | 309/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "archive room" | belief | - | 0.000035 | 337/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "wine cellar" | belief | - | 0.000036 | 351/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "north stairwell" | belief | - | 0.000036 | 349/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | A | ✓PASSanswered "archive room" | belief | - | 0.000035 | 345/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "west wing"unverified | belief | - | 0.000032 | 315/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "north stairwell" | belief | - | 0.000036 | 343/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "kitchen" | belief | - | 0.000034 | 329/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "storage room" | belief | - | 0.000036 | 353/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "utility closet"unverified | belief | - | 0.000034 | 328/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "security office"unverified | belief | - | 0.000035 | 344/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "courtyard"unverified | belief | - | 0.000035 | 345/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "workshop" | belief | - | 0.000034 | 334/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "cafeteria" | belief | - | 0.000033 | 319/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "photo studio" | belief | - | 0.000033 | 317/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "south stairwell"unverified | belief | - | 0.000033 | 313/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "workshop"unverified | belief | - | 0.000031 | 302/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "supply closet"unverified | belief | - | 0.000033 | 321/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "break room"unverified | belief | - | 0.000033 | 319/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "archive room" | belief | - | 0.000036 | 356/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "west wing"unverified | belief | - | 0.000032 | 309/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "print room"unverified | belief | - | 0.000037 | 358/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "front office" | belief | - | 0.000035 | 344/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "archive room"unverified | belief | - | 0.000034 | 327/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "supply closet" | belief | - | 0.000030 | 287/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "lobby"unverified | belief | - | 0.000035 | 347/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "meditation room"unverified | belief | - | 0.000033 | 322/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "supply closet" | belief | - | 0.000033 | 322/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "server room"unverified | belief | - | 0.000030 | 295/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "break room"unverified | belief | - | 0.000033 | 325/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "garage"unverified | belief | - | 0.000030 | 294/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "archive room"unverified | belief | - | 0.000033 | 322/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "wine cellar"unverified | belief | - | 0.000034 | 336/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ?FAILanswered "north stairwell"unverified | belief | - | 0.000035 | 334/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | B | ✓PASSanswered "security office" | belief | - | 0.000034 | 330/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "supply closet" | belief | - | 0.000051 | 503/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "north stairwell" | belief | - | 0.000058 | 567/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "kitchen" | belief | - | 0.000054 | 536/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "storage room" | belief | - | 0.000060 | 586/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "boiler room" | belief | - | 0.000054 | 529/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "archive room" | belief | - | 0.000058 | 569/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "workshop" | belief | - | 0.000058 | 572/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "workshop" | belief | - | 0.000055 | 543/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "cafeteria" | belief | - | 0.000053 | 514/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "photo studio" | belief | - | 0.000053 | 519/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ?FAILanswered "south stairwell"unverified | belief | - | 0.000052 | 505/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "print room" | belief | - | 0.000049 | 484/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ?FAILanswered "library"unverified | belief | - | 0.000053 | 522/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "boiler room" | belief | - | 0.000053 | 520/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "archive room" | belief | - | 0.000060 | 592/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ?FAILanswered "west wing"unverified | belief | - | 0.000050 | 490/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "gym" | belief | - | 0.000061 | 599/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "front office" | belief | - | 0.000058 | 567/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "south stairwell" | belief | - | 0.000054 | 529/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "supply closet" | belief | - | 0.000046 | 450/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "conference room" | belief | - | 0.000058 | 573/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "print room" | belief | - | 0.000053 | 522/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "supply closet" | belief | - | 0.000054 | 526/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "cafeteria" | belief | - | 0.000048 | 466/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "observation deck" | belief | - | 0.000054 | 529/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "photo studio" | belief | - | 0.000048 | 470/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "supply closet" | belief | - | 0.000053 | 524/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "greenhouse" | belief | - | 0.000056 | 550/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "garage" | belief | - | 0.000055 | 547/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | C | ✓PASSanswered "security office" | belief | - | 0.000055 | 538/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "west wing"unverified | belief | - | 0.000035 | 345/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "north stairwell" | belief | - | 0.000039 | 379/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "kitchen" | belief | - | 0.000037 | 362/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "storage room" | belief | - | 0.000040 | 392/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "utility closet"unverified | belief | - | 0.000037 | 361/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "archive room" | belief | - | 0.000039 | 383/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "workshop" | belief | - | 0.000039 | 384/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "photo studio"unverified | belief | - | 0.000038 | 370/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "cafeteria" | belief | - | 0.000036 | 352/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "photo studio" | belief | - | 0.000036 | 347/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "the kitchen" | belief | - | 0.000036 | 346/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "print room" | belief | - | 0.000034 | 332/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "basement" | belief | - | 0.000036 | 354/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "break room"unverified | belief | - | 0.000036 | 352/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "rooftop terrace"unverified | belief | - | 0.000041 | 395/5 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "west wing"unverified | belief | - | 0.000035 | 339/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "the gym" | belief | - | 0.000041 | 400/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "front office" | belief | - | 0.000039 | 383/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "archive room"unverified | belief | - | 0.000037 | 360/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "supply closet" | belief | - | 0.000032 | 314/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "lobby"unverified | belief | - | 0.000039 | 383/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "print room" | belief | - | 0.000036 | 355/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ✓PASSanswered "supply closet" | belief | - | 0.000036 | 355/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "server room"unverified | belief | - | 0.000033 | 322/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "break room"unverified | belief | - | 0.000037 | 361/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "garage"unverified | belief | - | 0.000033 | 324/2 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "archive room"unverified | belief | - | 0.000036 | 355/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "wine cellar"unverified | belief | - | 0.000038 | 372/3 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "north stairwell"unverified | belief | - | 0.000038 | 370/4 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | D | ?FAILanswered "archive room"unverified | belief | - | 0.000038 | 366/3 | full transcript → |