Data to …… trials
I · Narrative

Recovers the true order of events

Gemma 3 27B

Finding:HOLDS· grade B(research conclusion)

99% correct (79/80)

Tested up to chain length 20, scramble 1.0, distractor events 30

Pass ratesingle rate, no difficulty dial
99%

79/80 scored trials passed · 95% interval 93%–100%

breaks ≤ 30%holds if interval reaches 95%0%50%100%
What the few failures have in common

The model gives a wrong answer that matches no known shortcut.

in 100% of failures (1 failures)

Example failure
Trial result: FAIL (one model answer)
FAIL
Let's break down the timeline: * **8:00:** Wren brought the faded basket to the meditation room. * **8:01:** Quinn brought the faded basket to the basement. * **8:02:** Wren moved the cracked basket to the basement. * **8:03:** Wren brought the cracked basket to the meditation room. * **8:04:** Wren reorganized papers. * **8:05:** Wren refilled coffee. Quinn moving the faded basket to the basement
Evidencewhat this finding rests on
  • Ground truthDeterministic simulation: the right answer is computed, not judged
  • StatisticsWilson 95% interval 93%–100%Grade B: moderate evidence
  • Sample size80 scored trials in total
  • Fresh-seed confirmationNot run yet
  • ReplicationNot run yet
Raw evidenceruns and trials

Back to the capability map →