Data to …… trials

Run camp-full_map-gemma-001/breaking_point

57% PASS

What this run revealed, then every individual trial as evidence.

7
Trials
57%
4/7 scored
over trials
Pass rate
3
43%
Failures
$0.0004
Run cost

Run camp-full_map-gemma-001/breaking_point: 57% pass rate on belief queries.

Of 7 scored trials (belief query kind), the model passed 57% and failed 43%.

3 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

In 2 of 3 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.

Notable failures

  • record 7a3e678c — asked "Where does Cyrus believe the old box is? Answer with only the location name (or 'unknown' if they have no belief)."
    expected: the lobby
    actual: security office

    the model reported the object's TRUE final location, not what the queried entity actually believed -- it collapsed belief into reality

  • record ccc213d3 — asked "Where does Lucas believe the worn box is? Answer with only the location name (or 'unknown' if they have no belief)."
    expected: the greenhouse
    actual: rooftop terrace

    the model reported the object's TRUE final location, not what the queried entity actually believed -- it collapsed belief into reality

  • record f8e81c0a — asked "Where does Wyatt believe the blue wrench is? Answer with only the location name (or 'unknown' if they have no belief)."
    expected: the storage room
    actual: south stairwell

    the model's answer matches the 'query_agent_own_last_move' shortcut, not the verified belief

Trials7 trials
Per-trial table (7 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
google/gemma-3-27b-it✓PASSanswered "west wing"belief-0.000051501/3full transcript →
google/gemma-3-27b-it✓PASSanswered "mail room"belief-0.000049478/3full transcript →
google/gemma-3-27b-it?FAILanswered "security office"unverifiedbelief-0.000049483/3full transcript →
google/gemma-3-27b-it✓PASSanswered "conference room"belief-0.000048472/3full transcript →
google/gemma-3-27b-it?FAILanswered "rooftop terrace"unverifiedbelief-0.000052501/5full transcript →
google/gemma-3-27b-it✓PASSanswered "break room"belief-0.000051503/3full transcript →
google/gemma-3-27b-it?FAILanswered "south stairwell"unverifiedbelief-0.000050492/4full transcript →