III · Interactive execution
Executes a plan step by step, carrying values between calls (protocol v3)
Gemma 4 31B QAT
Finding:HOLDS· grade C(research conclusion)
Holds through 7-call plans, no values to carry: 5/5 pass across every tested level
Held through 7-call plans, no values to carry
Pass rate by level
Dashed lines: a level at or below 30% pass counts toward BREAKS, at or below 75% toward DEGRADES. Labels above points: passed/scored. Hollow points: fewer than 5 scored trials, not used for the call.
| Level | Passed | Pass rate | Note |
|---|---|---|---|
| 7-call plans, no values to carry | 5/5 | 100% | |
| 9-call plans, 1 value to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
| 11-call plans, 2 values to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
| 12-call plans, 4 values to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
| 14-call plans, 7 values to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
| 16-call plans, 10 values to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
| 18-call plans, 14 values to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
| 19-call plans, 16 values to carry | 4/4 | 100% | fewer than 5 scored, not used for the call |
What the few failures have in common
No failures recorded in the trials behind this row.
Evidence
- Ground truthDeterministic simulation: the right answer is computed, not judged
- StatisticsWilson 95% interval 57%–100%
- Sample size5 scored trials in total: small sample (under 15)
- Fresh-seed confirmationNot run yet
- ReplicationNot run yet
Raw evidence
- Runs behind this row
- Every interactive-execution trial for this model →