Run camp-full_map-gemma-001/belief
49% PASSWhat this run revealed, then every individual trial as evidence.
155
Trials
49%
76/155 scored
over trials
Pass rate
79
30 stable / 0 noise
Failures
$0.0038
Run cost
Run camp-full_map-gemma-001/belief: 49% pass rate on belief queries.
Of 155 scored trials (belief query kind), the model passed 49% and failed 51%.
79 failures recorded; 30 survived verification, 49 not yet verified.
In 63 of 79 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.
Notable failures
- record 78f187b6 expected: the lobbyactual: photo studio
- record 86ee5ed6 expected: the east wingactual: reception area
- record 1fd4fce9 expected: the boiler roomactual: wine cellar
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| google/gemma-3-27b-it | ?FAILanswered "photo studio"unverified | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "garage" | belief | - | 0.000023 | 219/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "supply closet" | belief | - | 0.000016 | 151/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000017 | 159/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "reception area"unverified | belief | - | 0.000016 | 150/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "wine cellar"unverified | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "kitchen" | belief | - | 0.000023 | 226/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "observation deck" | belief | - | 0.000024 | 232/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "basement" | belief | - | 0.000023 | 217/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "print room"unverified | belief | - | 0.000022 | 212/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "break room" | belief | - | 0.000026 | 251/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "reception area" | belief | - | 0.000029 | 281/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "break room" | belief | - | 0.000028 | 275/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "atrium"unverified | belief | - | 0.000029 | 283/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "boiler room" | belief | - | 0.000027 | 264/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "library" | belief | - | 0.000032 | 316/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "rooftop terrace"unverified | belief | - | 0.000037 | 359/5 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "kitchen"unverified | belief | - | 0.000036 | 352/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "garage"unverified | belief | - | 0.000036 | 354/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "cafeteria" | belief | - | 0.000039 | 374/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "reception area" | belief | - | 0.000035 | 342/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "rooftop terrace" | belief | - | 0.000036 | 349/5 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "courtyard"unverified | belief | - | 0.000036 | 353/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "mail room"unverified | belief | - | 0.000038 | 372/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "server room" | belief | - | 0.000037 | 364/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "photo studio" | belief | - | 0.000017 | 158/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "mail room"unverified | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "loading dock" | belief | - | 0.000018 | 171/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000016 | 148/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "security office" | belief | - | 0.000016 | 146/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "photo studio" | belief | - | 0.000018 | 166/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "library" | belief | - | 0.000016 | 151/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "loading dock" | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "rooftop terrace" | belief | - | 0.000018 | 163/5 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "east wing" | belief | - | 0.000017 | 161/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "rooftop terrace"unverified | belief | - | 0.000016 | 150/5 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "rooftop terrace" | belief | - | 0.000018 | 163/5 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "security office" | belief | - | 0.000018 | 168/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "greenhouse" | belief | - | 0.000017 | 161/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "storage room" | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "reception area"unverified | belief | - | 0.000017 | 163/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "wine cellar"unverified | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000016 | 152/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "front office" | belief | - | 0.000017 | 158/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "security office"unverified | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "observation deck" | belief | - | 0.000017 | 156/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "utility closet" | belief | - | 0.000018 | 168/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "archive room"unverified | belief | - | 0.000018 | 168/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "south stairwell"unverified | belief | - | 0.000018 | 165/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "photo studio"unverified | belief | - | 0.000018 | 166/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "courtyard"unverified | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "loading dock" | belief | - | 0.000018 | 167/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "courtyard"unverified | belief | - | 0.000017 | 158/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "rooftop terrace"unverified | belief | - | 0.000017 | 159/5 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "photo studio"unverified | belief | - | 0.000018 | 167/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "atrium" | belief | - | 0.000019 | 179/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "greenhouse"unverified | belief | - | 0.000019 | 178/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "library"unverified | belief | - | 0.000018 | 169/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000017 | 159/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "server room" | belief | - | 0.000017 | 165/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "photo studio"unverified | belief | - | 0.000019 | 182/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "basement"unverified | belief | - | 0.000018 | 171/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "meditation room"unverified | belief | - | 0.000020 | 189/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000021 | 197/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "security office"unverified | belief | - | 0.000023 | 222/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "supply closet"unverified | belief | - | 0.000022 | 214/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "east wing"unverified | belief | - | 0.000021 | 199/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "front office"unverified | belief | - | 0.000020 | 189/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000022 | 213/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "basement"unverified | belief | - | 0.000023 | 216/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000024 | 229/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "boiler room" | belief | - | 0.000024 | 232/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "archive room"unverified | belief | - | 0.000025 | 236/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "garage"unverified | belief | - | 0.000023 | 227/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "conference room" | belief | - | 0.000024 | 232/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "lobby"unverified | belief | - | 0.000024 | 233/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "server room" | belief | - | 0.000016 | 154/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "front office"unverified | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown"unverified | belief | - | 0.000017 | 166/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000017 | 163/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "basement"unverified | belief | - | 0.000017 | 159/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown"unverified | belief | - | 0.000022 | 218/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "archive room" | belief | - | 0.000024 | 228/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "security office" | belief | - | 0.000023 | 221/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "east wing"unverified | belief | - | 0.000034 | 325/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "library"unverified | belief | - | 0.000031 | 300/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "photo studio" | belief | - | 0.000024 | 227/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown"unverified | belief | - | 0.000042 | 408/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "loading dock"unverified | belief | - | 0.000043 | 418/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown"unverified | belief | - | 0.000033 | 322/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "kitchen"unverified | belief | - | 0.000041 | 401/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000032 | 310/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "basement" | belief | - | 0.000032 | 306/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "meditation room" | belief | - | 0.000051 | 493/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000049 | 479/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000042 | 408/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown"unverified | belief | - | 0.000051 | 504/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "storage room" | belief | - | 0.000043 | 417/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "reception area" | belief | - | 0.000049 | 475/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown"unverified | belief | - | 0.000049 | 484/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "kitchen"unverified | belief | - | 0.000015 | 148/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "kitchen" | belief | - | 0.000017 | 165/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "workshop"unverified | belief | - | 0.000017 | 164/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "west wing"verified-stable | belief | - | 0.000017 | 165/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "break room"verified-stable | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "photo studio"verified-stable | belief | - | 0.000017 | 160/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "library"verified-stable | belief | - | 0.000017 | 163/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "break room"verified-stable | belief | - | 0.000016 | 150/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "library" | belief | - | 0.000016 | 150/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "library"verified-stable | belief | - | 0.000017 | 164/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000017 | 166/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "courtyard"verified-stable | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "library" | belief | - | 0.000015 | 145/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "gym"verified-stable | belief | - | 0.000017 | 163/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "security office"verified-stable | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "mail room" | belief | - | 0.000016 | 147/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "boiler room" | belief | - | 0.000017 | 156/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "observation deck" | belief | - | 0.000016 | 147/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "storage room" | belief | - | 0.000016 | 146/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000018 | 167/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "loading dock"verified-stable | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "greenhouse"verified-stable | belief | - | 0.000017 | 159/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "courtyard" | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "utility closet" | belief | - | 0.000016 | 151/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "workshop"verified-stable | belief | - | 0.000015 | 146/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "archive room"verified-stable | belief | - | 0.000037 | 360/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "loading dock" | belief | - | 0.000036 | 353/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "server room"verified-stable | belief | - | 0.000034 | 333/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "west wing"verified-stable | belief | - | 0.000036 | 349/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "break room"verified-stable | belief | - | 0.000037 | 360/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "boiler room"verified-stable | belief | - | 0.000020 | 186/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "east wing" | belief | - | 0.000020 | 190/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "photo studio"verified-stable | belief | - | 0.000020 | 190/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "loading dock"verified-stable | belief | - | 0.000020 | 186/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "lobby"verified-stable | belief | - | 0.000019 | 186/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000020 | 190/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "courtyard"verified-stable | belief | - | 0.000020 | 188/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "loading dock"verified-stable | belief | - | 0.000020 | 192/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "library"verified-stable | belief | - | 0.000019 | 187/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "west wing" | belief | - | 0.000020 | 188/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "east wing" | belief | - | 0.000034 | 335/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000034 | 327/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000032 | 309/4 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "storage room"verified-stable | belief | - | 0.000032 | 315/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown"verified-stable | belief | - | 0.000033 | 320/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "kitchen" | belief | - | 0.000036 | 355/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown"verified-stable | belief | - | 0.000036 | 351/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "greenhouse" | belief | - | 0.000038 | 367/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "archive room" | belief | - | 0.000039 | 382/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "security office" | belief | - | 0.000040 | 389/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown"verified-stable | belief | - | 0.000040 | 397/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "photo studio"verified-stable | belief | - | 0.000041 | 399/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "greenhouse" | belief | - | 0.000039 | 381/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "archive room"verified-stable | belief | - | 0.000040 | 392/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "storage room"verified-stable | belief | - | 0.000041 | 403/3 | full transcript → |