Data to …… trials
I · Narrative

Tracks where things actually are

Nemotron 3 Super 120B

Finding:HOLDS· grade B(research conclusion)

96% correct (91/95)

Tested up to chain length 30, distractor events 2

Pass ratesingle rate, no difficulty dial
96%

91/95 scored trials passed · 95% interval 90%–98%

breaks ≤ 30%holds if interval reaches 95%0%50%100%
What the few failures have in common

The model gives a wrong answer that matches no known shortcut.

in 100% of failures (4 failures)

Example failure
Trial result: FAIL (one model answer)
FAIL
wine cellar
Open this trial →Open the run it came from →
Evidencewhat this finding rests on
  • Ground truthDeterministic simulation: the right answer is computed, not judged
  • StatisticsWilson 95% interval 90%–98%Grade B: moderate evidence
  • Sample size95 scored trials in total
  • Fresh-seed confirmationNot run yet
  • ReplicationNot run yet
Raw evidenceruns and trials

Back to the capability map →