Data to …… trials
I · Narrative

Recovers the true order of events

Gemma 4 31B QAT

Finding:HOLDS· grade A(research conclusion)

95% correct (199/209)

Tested up to chain length 20, scramble 1.0, distractor events 30

Pass ratesingle rate, no difficulty dial
95%

199/209 scored trials passed · 95% interval 91%–97%

breaks ≤ 30%holds if interval reaches 95%0%50%100%
What the few failures have in common

The model gives a wrong answer that matches no known shortcut.

in 100% of failures (10 failures)

Example failure
Trial result: FAIL (one model answer)
FAIL
Based on the text provided, there is no mention of "Wren's aside." Additionally, the text states that "she" (referring to Ingrid, as she is the only female mentioned besides Wren, and the sequence of events with the folder involves Ingrid) took the cracked folder to the mail room at 8:00, not Wren. Since neither event described in your question actually occurred as stated, the question cannot be a
Evidencewhat this finding rests on
  • Ground truthDeterministic simulation: the right answer is computed, not judged
  • StatisticsWilson 95% interval 91%–97%Grade A: strong evidence
  • Sample size209 scored trials in total
  • Fresh-seed confirmationNot run yet
  • ReplicationNot run yet
Raw evidenceruns and trials

Back to the capability map →