Run camp-full_map-nvidia-nemotron-3-nano-30b-a3b-001/breaking_point

Run camp-full_map-nvidia-nemotron-3-nano-30b-a3b-001/breaking_point: 87% pass rate on belief queries.

Of 45 scored trials (belief query kind), the model passed 87% and failed 13%.

11 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.

6 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Diffuse weakness (below 90% but no outright flip) on: context_length (pass rate 74% at the high end), coreference (pass rate 87% at the high end), distractors (pass rate 74% at the high end), entity_state (pass rate 74% at the high end), temporal (pass rate 76% at the high end), witness_gap (pass rate 74% at the high end).

6 belief failure(s) were classified against known shortcuts, but none of them dominate -- the failures don't share one obvious wrong-answer pattern.

Notable failures

• record 028072fd — asked "Where does Rosa believe the bright wrench is? Answer with only the location name (or 'unknown' if they have no belief)."
  expected: the print room
  actual: workshop

  the model reported the object's TRUE final location, not what the queried entity actually believed -- it collapsed belief into reality

• record df5c17c4 — asked "Where does Wyatt believe the large box is? Answer with only the location name (or 'unknown' if they have no belief)."
  expected: the archive room
  actual: storage room

  the model's answer matches the 'last_mentioned_location' shortcut, not the verified belief

• record 6e8fcb70 — asked "Where does Juno believe the faded wrench is? Answer with only the location name (or 'unknown' if they have no belief)."
  expected: the rooftop terrace
  actual: security office

  the model's answer matches no known shortcut -- an unclassified miss