Data to …… trials
I · Narrative

Belief tracking in deliberately misleading stories

Nemotron 3 Ultra 550B

Finding:HOLDS· grade A(research conclusion)

99% correct (127/128)

Tested up to chain length 10, scramble 1.0, pronoun pressure 5, distractor events 20, witness cutoff 0.9

Pass ratesingle rate, no difficulty dial
99%

127/128 scored trials passed · 95% interval 96%–100%

breaks ≤ 30%holds if interval reaches 95%0%50%100%
What the few failures have in common

The model answers 'unknown' when the belief can be worked out.

in 100% of failures (1 failures)

Example failure
Trial result: FAIL (one model answer)
FAIL
unknown
Open this trial →Open the run it came from →
Evidencewhat this finding rests on
  • Ground truthDeterministic simulation: the right answer is computed, not judged
  • StatisticsWilson 95% interval 96%–100%Grade A: strong evidence
  • Sample size128 scored trials in total2 errored trials excluded
  • Fresh-seed confirmationNot run yet
  • ReplicationNot run yet
Raw evidenceruns and trials

Back to the capability map →