Data to …… trials

Run camp-full_map-gemma-001

57% PASS

What this run revealed, then every individual trial as evidence.

432
Trials
57%
246/432 scored
over trials
Pass rate
186
81 stable / 0 noise
43%
Failures
$0.0145
Run cost

Full-map run camp-full_map-gemma-001: clean on order, breaking_point, weak province: state, belief, adversarial_belief.

State is a weak province: 56% pass rate across 45 trials.

(state) 20 failures recorded; all 20 survived verification.

Passes order cleanly (99%, 80 trials).

(order) 1 failures recorded; all 1 survived verification.

Belief is a weak province: 49% pass rate across 155 trials.

(belief) 79 failures recorded; 30 survived verification, 49 not yet verified.

(belief) In 63 of 79 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.

Adversarial_belief is a weak province: 43% pass rate across 145 trials.

(adversarial_belief) 83 failures recorded; 30 survived verification, 53 not yet verified.

(adversarial_belief) In 74 of 83 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.

breaking_point: escalation ray probed with 4 pass / 3 fail across 7 trials (failures not yet verified).

Notable failures

  • record d55ca622 — asked "Where is the small box now? Answer with only the location name."
    expected: the boiler room
    actual: West wing

    the model's tracked object state diverged from the simulated ground truth

  • record 9bc3cedf — asked "Where is the small wrench now? Answer with only the location name."
    expected: the east wing
    actual: mail room

    the model's tracked object state diverged from the simulated ground truth

  • record d73cbb84 — asked "Which happened first: Quinn moving the faded basket to the meditation room or Wren's aside? Answer 'first' if the former, 'second' if the latter."
    expected: first
    actual: **second**

    the model recovered narration order, not the true event timestamps

Trials432 trials
Per-trial table (432 trials -- prompt, reasoning, verdict, answer key)
modelstageverdictkinddifficultycost ($)tokens in/out
google/gemma-3-27b-itstate✓PASSanswered "Utility closet"state-0.000014128/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Courtyard"state-0.000013125/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "West wing"verified-stablestate-0.000027256/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Photo studio"state-0.000013125/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Storage room"state-0.000013125/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Courtyard"state-0.000013122/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Cafeteria"state-0.000026247/4full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Storage room"state-0.000025244/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Observation deck"state-0.000026249/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "loading dock"state-0.000038371/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Boiler room"state-0.000026249/4full transcript →
google/gemma-3-27b-itstate✕FAILanswered "mail room"verified-stablestate-0.000039376/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Cafeteria"state-0.000039380/4full transcript →
google/gemma-3-27b-itstate✓PASSanswered "mail room"state-0.000038371/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Server room"verified-stablestate-0.000040388/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "West wing"state-0.000050493/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "The archive room."state-0.000051492/5full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Wine cellar"verified-stablestate-0.000050488/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Security office"verified-stablestate-0.000052511/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Print room"state-0.000051504/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Storage room"state-0.000064627/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "atrium"verified-stablestate-0.000063617/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "rooftop terrace"state-0.000063617/5full transcript →
google/gemma-3-27b-itstate✕FAILanswered "wine cellar"verified-stablestate-0.000065641/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Archive room"verified-stablestate-0.000064630/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Photo studio"state-0.000057562/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "north stairwell"verified-stablestate-0.000056546/4full transcript →
google/gemma-3-27b-itstate✓PASSanswered "print room"state-0.000056554/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "library"state-0.000056555/2full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Courtyard"verified-stablestate-0.000057559/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Garage"verified-stablestate-0.000060589/2full transcript →
google/gemma-3-27b-itstate✕FAILanswered "photo studio"verified-stablestate-0.000060586/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Kitchen"verified-stablestate-0.000059583/2full transcript →
google/gemma-3-27b-itstate✓PASSanswered "West wing"state-0.000060591/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "mail room"state-0.000061598/3full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Workshop"state-0.000058573/2full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Storage room"verified-stablestate-0.000058569/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "library"verified-stablestate-0.000058576/2full transcript →
google/gemma-3-27b-itstate✕FAILanswered "server room"verified-stablestate-0.000057561/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "break room"verified-stablestate-0.000057561/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Library"verified-stablestate-0.000062610/2full transcript →
google/gemma-3-27b-itstate✓PASSanswered "workshop"state-0.000062616/2full transcript →
google/gemma-3-27b-itstate✓PASSanswered "Storage room"state-0.000064627/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "boiler room"verified-stablestate-0.000064634/3full transcript →
google/gemma-3-27b-itstate✕FAILanswered "Gym"verified-stablestate-0.000065646/2full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, Bruno moving the box happened first."order-0.000033149/59full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:04, so Oscar's action happened first."order-0.000034146/66full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000035142/69full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000048213/90full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, Maya's action happened first."order-0.000032152/56full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000056219/113full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:04, so Sana moving the tablet to the bre..."order-0.000035144/69full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000053216/103full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:08, so Noor's action happened first."order-0.000043218/69full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000047299/56full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000058302/94full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000064313/108full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000052298/74full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000051311/66full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000066391/91full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000069394/98full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000108217/289full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**first**"order-0.000072469/83full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000077472/99full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, Kira moving the box happened first."order-0.000064461/60full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000061401/69full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000074467/90full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000069400/97full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000081461/116full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000075404/115full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**first**"order-0.000034143/64full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000039146/81full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000042145/90full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000041140/91full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:05, so moving the box happened first."order-0.000031146/56full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Dana's action happened at 8:00, which is before Wyatt's a..."order-0.000038146/79full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000069149/180full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000047143/108full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000067146/174full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000079151/214full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000065143/170full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000069149/179full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000078147/210full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000035138/72full transcript →
google/gemma-3-27b-itorder✓PASSanswered "first"order-0.000015141/3full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000068142/179full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000068150/176full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**first**"order-0.000060148/151full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**first**"order-0.000064149/164full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000082145/226full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Since 8:00 is earlier than 8:05, Sana moving the folder h..."order-0.000036152/68full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000064142/167full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000066150/170full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000069148/182full transcript →
google/gemma-3-27b-itorder✕FAILanswered "**second**"verified-stableorder-0.000062143/159full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:04, so moving the folder happened first."order-0.000034147/63full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000037142/76full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000050263/79full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000043142/97full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000041280/45full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000047272/65full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000054280/86full transcript →
google/gemma-3-27b-itorder✓PASSanswered "The answer is **first**."order-0.000039148/79full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000075274/160full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**first**"order-0.000046143/105full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000063386/83full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000071401/102full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000067394/92full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: first"order-0.000065381/89full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000082506/106full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000081513/98full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000091543/122full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000078524/85full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:32, so Willa moving the box happened first."order-0.000087653/71full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000088627/84full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000077526/83full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: **first**"order-0.000093651/93full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000101660/115full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000068394/94full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Answer: first"order-0.000088612/90full transcript →
google/gemma-3-27b-itorder✓PASSanswered "first"order-0.000016147/3full transcript →
google/gemma-3-27b-itorder✓PASSanswered "8:00 is before 8:05, so Finn moving the folder to the uti..."order-0.000033148/62full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000047146/109full transcript →
google/gemma-3-27b-itorder✓PASSanswered "**Answer: first**"order-0.000036152/71full transcript →
google/gemma-3-27b-itorder✓PASSanswered "Therefore, the answer is **first**."order-0.000039144/81full transcript →
google/gemma-3-27b-itbelief?FAILanswered "photo studio"unverifiedbelief-0.000016148/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "garage"belief-0.000023219/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "supply closet"belief-0.000016151/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "workshop"belief-0.000017159/2full transcript →
google/gemma-3-27b-itbelief?FAILanswered "reception area"unverifiedbelief-0.000016150/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "wine cellar"unverifiedbelief-0.000017162/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "kitchen"belief-0.000023226/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "observation deck"belief-0.000024232/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "basement"belief-0.000023217/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "print room"unverifiedbelief-0.000022212/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "break room"belief-0.000026251/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "reception area"belief-0.000029281/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "break room"belief-0.000028275/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "atrium"unverifiedbelief-0.000029283/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "boiler room"belief-0.000027264/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "library"belief-0.000032316/2full transcript →
google/gemma-3-27b-itbelief?FAILanswered "rooftop terrace"unverifiedbelief-0.000037359/5full transcript →
google/gemma-3-27b-itbelief?FAILanswered "kitchen"unverifiedbelief-0.000036352/2full transcript →
google/gemma-3-27b-itbelief?FAILanswered "garage"unverifiedbelief-0.000036354/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "cafeteria"belief-0.000039374/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "reception area"belief-0.000035342/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "rooftop terrace"belief-0.000036349/5full transcript →
google/gemma-3-27b-itbelief?FAILanswered "courtyard"unverifiedbelief-0.000036353/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "mail room"unverifiedbelief-0.000038372/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "server room"belief-0.000037364/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "photo studio"belief-0.000017158/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "mail room"unverifiedbelief-0.000017162/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "loading dock"belief-0.000018171/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "north stairwell"belief-0.000016148/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "security office"belief-0.000016146/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "photo studio"belief-0.000018166/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "library"belief-0.000016151/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "loading dock"belief-0.000016148/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "rooftop terrace"belief-0.000018163/5full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "east wing"belief-0.000017161/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "rooftop terrace"unverifiedbelief-0.000016150/5full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "rooftop terrace"belief-0.000018163/5full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "security office"belief-0.000018168/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "greenhouse"belief-0.000017161/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "storage room"belief-0.000016149/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "reception area"unverifiedbelief-0.000017163/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "wine cellar"unverifiedbelief-0.000016148/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "workshop"belief-0.000016152/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "front office"belief-0.000017158/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "security office"unverifiedbelief-0.000017162/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "observation deck"belief-0.000017156/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "utility closet"belief-0.000018168/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "archive room"unverifiedbelief-0.000018168/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "south stairwell"unverifiedbelief-0.000018165/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "photo studio"unverifiedbelief-0.000018166/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "courtyard"unverifiedbelief-0.000016148/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "loading dock"belief-0.000018167/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "courtyard"unverifiedbelief-0.000017158/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "rooftop terrace"unverifiedbelief-0.000017159/5full transcript →
google/gemma-3-27b-itbelief?FAILanswered "photo studio"unverifiedbelief-0.000018167/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "atrium"belief-0.000019179/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "greenhouse"unverifiedbelief-0.000019178/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "library"unverifiedbelief-0.000018169/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "workshop"belief-0.000017159/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "server room"belief-0.000017165/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "photo studio"unverifiedbelief-0.000019182/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "basement"unverifiedbelief-0.000018171/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "meditation room"unverifiedbelief-0.000020189/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "south stairwell"belief-0.000021197/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "security office"unverifiedbelief-0.000023222/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "supply closet"unverifiedbelief-0.000022214/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "east wing"unverifiedbelief-0.000021199/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "front office"unverifiedbelief-0.000020189/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "workshop"belief-0.000022213/2full transcript →
google/gemma-3-27b-itbelief?FAILanswered "basement"unverifiedbelief-0.000023216/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "north stairwell"belief-0.000024229/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "boiler room"belief-0.000024232/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "archive room"unverifiedbelief-0.000025236/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "garage"unverifiedbelief-0.000023227/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "conference room"belief-0.000024232/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "lobby"unverifiedbelief-0.000024233/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "server room"belief-0.000016154/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "front office"unverifiedbelief-0.000017162/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "unknown"unverifiedbelief-0.000017166/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "south stairwell"belief-0.000017163/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "basement"unverifiedbelief-0.000017159/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "unknown"unverifiedbelief-0.000022218/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "archive room"belief-0.000024228/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "security office"belief-0.000023221/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "east wing"unverifiedbelief-0.000034325/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "library"unverifiedbelief-0.000031300/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "photo studio"belief-0.000024227/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "unknown"unverifiedbelief-0.000042408/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "loading dock"unverifiedbelief-0.000043418/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "unknown"unverifiedbelief-0.000033322/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "kitchen"unverifiedbelief-0.000041401/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "north stairwell"belief-0.000032310/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "basement"belief-0.000032306/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "meditation room"belief-0.000051493/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "wine cellar"belief-0.000049479/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "south stairwell"belief-0.000042408/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "unknown"unverifiedbelief-0.000051504/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "storage room"belief-0.000043417/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "reception area"belief-0.000049475/4full transcript →
google/gemma-3-27b-itbelief?FAILanswered "unknown"unverifiedbelief-0.000049484/3full transcript →
google/gemma-3-27b-itbelief?FAILanswered "kitchen"unverifiedbelief-0.000015148/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "kitchen"belief-0.000017165/2full transcript →
google/gemma-3-27b-itbelief?FAILanswered "workshop"unverifiedbelief-0.000017164/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "west wing"verified-stablebelief-0.000017165/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "break room"verified-stablebelief-0.000017162/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "photo studio"verified-stablebelief-0.000017160/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "library"verified-stablebelief-0.000017163/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "break room"verified-stablebelief-0.000016150/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "library"belief-0.000016150/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "library"verified-stablebelief-0.000017164/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "workshop"belief-0.000017166/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "courtyard"verified-stablebelief-0.000016149/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "library"belief-0.000015145/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "gym"verified-stablebelief-0.000017163/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "security office"verified-stablebelief-0.000016149/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "mail room"belief-0.000016147/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "boiler room"belief-0.000017156/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "observation deck"belief-0.000016147/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "storage room"belief-0.000016146/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "wine cellar"belief-0.000018167/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "loading dock"verified-stablebelief-0.000016149/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "greenhouse"verified-stablebelief-0.000017159/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "courtyard"belief-0.000016148/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "utility closet"belief-0.000016151/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "workshop"verified-stablebelief-0.000015146/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "archive room"verified-stablebelief-0.000037360/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "loading dock"belief-0.000036353/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "server room"verified-stablebelief-0.000034333/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "west wing"verified-stablebelief-0.000036349/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "break room"verified-stablebelief-0.000037360/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "boiler room"verified-stablebelief-0.000020186/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "east wing"belief-0.000020190/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "photo studio"verified-stablebelief-0.000020190/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "loading dock"verified-stablebelief-0.000020186/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "lobby"verified-stablebelief-0.000019186/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "workshop"belief-0.000020190/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "courtyard"verified-stablebelief-0.000020188/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "loading dock"verified-stablebelief-0.000020192/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "library"verified-stablebelief-0.000019187/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "west wing"belief-0.000020188/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "east wing"belief-0.000034335/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "north stairwell"belief-0.000034327/4full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "south stairwell"belief-0.000032309/4full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "storage room"verified-stablebelief-0.000032315/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "unknown"verified-stablebelief-0.000033320/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "kitchen"belief-0.000036355/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "unknown"verified-stablebelief-0.000036351/2full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "greenhouse"belief-0.000038367/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "archive room"belief-0.000039382/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "security office"belief-0.000040389/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "unknown"verified-stablebelief-0.000040397/2full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "photo studio"verified-stablebelief-0.000041399/3full transcript →
google/gemma-3-27b-itbelief✓PASSanswered "greenhouse"belief-0.000039381/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "archive room"verified-stablebelief-0.000040392/3full transcript →
google/gemma-3-27b-itbelief✕FAILanswered "storage room"verified-stablebelief-0.000041403/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "front office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000016148/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017165/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000015146/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "workshop" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000021203/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017161/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "north stairwell"belief-0.000017155/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "north stairwell"belief-0.000023214/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "storage room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000023216/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000028270/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "observation deck" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000027259/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000020197/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "break room"belief-0.000022209/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "archive room"belief-0.000028272/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "break room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000034327/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000029277/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "conference room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000030287/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000036348/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "greenhouse" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000034335/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "meditation room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000031303/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000032312/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "mail room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000039383/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "wine cellar"belief-0.000038371/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "lobby"belief-0.000042410/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "workshop" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000043427/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone, most_frequent_location, query_agent_own_last_move)unverifiedbelief-0.000037367/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "utility closet"belief-0.000016148/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "front office"belief-0.000017163/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "observation deck"belief-0.000016147/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017161/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "storage room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017159/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017164/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "print room"belief-0.000016149/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000018168/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "break room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017165/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "supply closet"belief-0.000016147/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "mail room"belief-0.000017165/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "photo studio"belief-0.000016155/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "workshop" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000015143/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "north stairwell"belief-0.000018165/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "the garage"belief-0.000016155/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "photo studio" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000018172/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000018168/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "library"belief-0.000015144/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "boiler room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017161/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "archive room"belief-0.000016148/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "photo studio" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017162/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "break room"belief-0.000018168/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "gym"belief-0.000015148/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017164/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "rooftop terrace"belief-0.000016149/5full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "conference room"belief-0.000018166/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "atrium"belief-0.000017161/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "photo studio"belief-0.000016149/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "courtyard"belief-0.000017163/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "cafeteria" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017160/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "gym" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017160/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "atrium" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000019177/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "atrium"belief-0.000017163/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "break room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000019180/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "loading dock"belief-0.000019177/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "mail room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000020189/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "south stairwell"belief-0.000020192/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "east wing" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000020187/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "utility closet"belief-0.000020187/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000022213/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "loading dock" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000023220/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "front office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000021200/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000020186/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "south stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000024230/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "greenhouse"belief-0.000024231/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000023216/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "wine cellar"belief-0.000024227/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "west wing" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000025238/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000022215/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "meditation room"belief-0.000025234/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000017164/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "meditation room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000018164/4full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "wine cellar" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000018171/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000024232/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000018166/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "south stairwell"belief-0.000016148/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "mail room"belief-0.000025239/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "archive room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000023220/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000033320/3full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000024232/2full transcript →
google/gemma-3-27b-itadversarial_belief?FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)unverifiedbelief-0.000032315/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "north stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000023221/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "garage"belief-0.000040395/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "archive room"belief-0.000035337/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000033320/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "server room"belief-0.000042406/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "security office"belief-0.000031302/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000042409/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000051499/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "west wing"belief-0.000043414/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "gym"belief-0.000041402/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "east wing"belief-0.000048468/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "storage room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000049484/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "cafeteria" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000049481/4full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000048474/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "workshop"belief-0.000016149/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "lobby" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000017161/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "security office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000016150/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "greenhouse"belief-0.000016147/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "break room"belief-0.000017164/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "wine cellar"belief-0.000016149/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "archive room"belief-0.000017163/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000017163/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "security office"belief-0.000016150/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "server room"belief-0.000016150/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "workshop"belief-0.000017163/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "cafeteria" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000018167/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "wine cellar"belief-0.000017157/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "basement"belief-0.000016155/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "front office" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000016149/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "boiler room"belief-0.000017156/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "library" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000017163/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "boiler room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000016150/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "north stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000018166/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "wine cellar"belief-0.000016151/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "wine cellar" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000017160/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000016152/4full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "courtyard" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000017159/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "conference room"belief-0.000017162/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "east wing"belief-0.000018167/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "garage"belief-0.000020191/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "south stairwell"belief-0.000021193/4full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "loading dock" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000021198/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "basement"belief-0.000019176/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "photo studio" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000019182/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "kitchen" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000021199/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000022208/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "reception area" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000022205/4full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "atrium" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000022203/4full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "conference room"belief-0.000021202/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "basement" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000045445/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "loading dock"belief-0.000046450/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "conference room"belief-0.000045443/3full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "wine cellar"belief-0.000046447/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000046449/2full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "north stairwell" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000049478/4full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "unknown" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000047466/2full transcript →
google/gemma-3-27b-itadversarial_belief✓PASSanswered "basement"belief-0.000047459/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "server room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000048467/3full transcript →
google/gemma-3-27b-itadversarial_belief✕FAILanswered "print room" (world seeded for: final_true_state, last_mentioned_location, last_witnessed_by_anyone)verified-stablebelief-0.000046451/3full transcript →
google/gemma-3-27b-itbreaking_point✓PASSanswered "west wing"belief-0.000051501/3full transcript →
google/gemma-3-27b-itbreaking_point✓PASSanswered "mail room"belief-0.000049478/3full transcript →
google/gemma-3-27b-itbreaking_point?FAILanswered "security office"unverifiedbelief-0.000049483/3full transcript →
google/gemma-3-27b-itbreaking_point✓PASSanswered "conference room"belief-0.000048472/3full transcript →
google/gemma-3-27b-itbreaking_point?FAILanswered "rooftop terrace"unverifiedbelief-0.000052501/5full transcript →
google/gemma-3-27b-itbreaking_point✓PASSanswered "break room"belief-0.000051503/3full transcript →
google/gemma-3-27b-itbreaking_point?FAILanswered "south stairwell"unverifiedbelief-0.000050492/4full transcript →