Run camp-full_map-gemma-001/adversarial_belief
43% PASSWhat this run revealed, then every individual trial as evidence.
145
Trials
43%
62/145 scored
over trials
Pass rate
83
30 stable / 0 noise
Failures
$0.0035
Run cost
Run camp-full_map-gemma-001/adversarial_belief: 43% pass rate on belief queries.
Of 145 scored trials (belief query kind), the model passed 43% and failed 57%.
83 failures recorded; 30 survived verification, 53 not yet verified.
In 74 of 83 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.
Notable failures
- record 28d3fb97 expected: the mail roomactual: front office
- record 8df09675 expected: the supply closetactual: print room
- record 26a3225f expected: the utility closetactual: kitchen
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| google/gemma-3-27b-it | ?FAILanswered "front office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 165/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000015 | 146/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "workshop" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000021 | 203/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 161/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000017 | 155/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000023 | 214/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "storage room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000023 | 216/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000028 | 270/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "observation deck" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000027 | 259/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000020 | 197/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "break room" | belief | - | 0.000022 | 209/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "archive room" | belief | - | 0.000028 | 272/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "break room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000034 | 327/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000029 | 277/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "conference room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000030 | 287/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000036 | 348/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "greenhouse" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000034 | 335/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "meditation room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000031 | 303/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000032 | 312/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "mail room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000039 | 383/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000038 | 371/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "lobby" | belief | - | 0.000042 | 410/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "workshop" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000043 | 427/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverified | belief | - | 0.000037 | 367/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "utility closet" | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "front office" | belief | - | 0.000017 | 163/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "observation deck" | belief | - | 0.000016 | 147/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 161/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "storage room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 159/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 164/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "print room" | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000018 | 168/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "break room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 165/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "supply closet" | belief | - | 0.000016 | 147/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "mail room" | belief | - | 0.000017 | 165/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "photo studio" | belief | - | 0.000016 | 155/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "workshop" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000015 | 143/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "north stairwell" | belief | - | 0.000018 | 165/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "the garage" | belief | - | 0.000016 | 155/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "photo studio" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000018 | 172/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000018 | 168/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "library" | belief | - | 0.000015 | 144/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "boiler room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 161/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "archive room" | belief | - | 0.000016 | 148/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "photo studio" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "break room" | belief | - | 0.000018 | 168/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "gym" | belief | - | 0.000015 | 148/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 164/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "rooftop terrace" | belief | - | 0.000016 | 149/5 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "conference room" | belief | - | 0.000018 | 166/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "atrium" | belief | - | 0.000017 | 161/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "photo studio" | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "courtyard" | belief | - | 0.000017 | 163/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "cafeteria" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 160/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "gym" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 160/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "atrium" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000019 | 177/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "atrium" | belief | - | 0.000017 | 163/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "break room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000019 | 180/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "loading dock" | belief | - | 0.000019 | 177/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "mail room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000020 | 189/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000020 | 192/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "east wing" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000020 | 187/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "utility closet" | belief | - | 0.000020 | 187/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000022 | 213/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "loading dock" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000023 | 220/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "front office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000021 | 200/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000020 | 186/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000024 | 230/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "greenhouse" | belief | - | 0.000024 | 231/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000023 | 216/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000024 | 227/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "west wing" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000025 | 238/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000022 | 215/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "meditation room" | belief | - | 0.000025 | 234/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000017 | 164/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "meditation room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000018 | 164/4 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "wine cellar" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000018 | 171/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000024 | 232/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000018 | 166/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000016 | 148/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "mail room" | belief | - | 0.000025 | 239/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000023 | 220/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000033 | 320/3 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000024 | 232/2 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverified | belief | - | 0.000032 | 315/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "north stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000023 | 221/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "garage" | belief | - | 0.000040 | 395/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "archive room" | belief | - | 0.000035 | 337/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000033 | 320/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "server room" | belief | - | 0.000042 | 406/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "security office" | belief | - | 0.000031 | 302/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000042 | 409/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000051 | 499/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "west wing" | belief | - | 0.000043 | 414/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "gym" | belief | - | 0.000041 | 402/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "east wing" | belief | - | 0.000048 | 468/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "storage room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000049 | 484/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "cafeteria" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000049 | 481/4 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000048 | 474/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000016 | 149/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "lobby" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000017 | 161/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "security office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000016 | 150/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "greenhouse" | belief | - | 0.000016 | 147/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "break room" | belief | - | 0.000017 | 164/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "archive room" | belief | - | 0.000017 | 163/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000017 | 163/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "security office" | belief | - | 0.000016 | 150/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "server room" | belief | - | 0.000016 | 150/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "workshop" | belief | - | 0.000017 | 163/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "cafeteria" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000018 | 167/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000017 | 157/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "basement" | belief | - | 0.000016 | 155/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "front office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000016 | 149/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "boiler room" | belief | - | 0.000017 | 156/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000017 | 163/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "boiler room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000016 | 150/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "north stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000018 | 166/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000016 | 151/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "wine cellar" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000017 | 160/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000016 | 152/4 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "courtyard" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000017 | 159/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "conference room" | belief | - | 0.000017 | 162/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "east wing" | belief | - | 0.000018 | 167/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "garage" | belief | - | 0.000020 | 191/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "south stairwell" | belief | - | 0.000021 | 193/4 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "loading dock" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000021 | 198/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "basement" | belief | - | 0.000019 | 176/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "photo studio" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000019 | 182/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000021 | 199/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000022 | 208/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000022 | 205/4 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "atrium" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000022 | 203/4 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "conference room" | belief | - | 0.000021 | 202/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000045 | 445/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "loading dock" | belief | - | 0.000046 | 450/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "conference room" | belief | - | 0.000045 | 443/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "wine cellar" | belief | - | 0.000046 | 447/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000046 | 449/2 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "north stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000049 | 478/4 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000047 | 466/2 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "basement" | belief | - | 0.000047 | 459/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "server room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000048 | 467/3 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stable | belief | - | 0.000046 | 451/3 | full transcript → |