Run camp-code-repair-confirm-gemma4qat-001
35% PASSWhat this run revealed, then every individual trial as evidence.
20
3 errors
Trials
35%
6/17 scored
over trials
Pass rate
11
Failures
$0.0076
Run cost
Run camp-code-repair-confirm-gemma4qat-001, 1 arms: clean on none; below 97% on trace.
Arm trace: 65% pass rate across 20 trials.
(trace) 7 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 31650d9e expected: (unknown)actual: ANSWER: 13
- record 29d29945 expected: (unknown)actual: ANSWER: [1, 2, 0, 8, 1, -2, 12, 15, 3, 1, 2]
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000162 | 366/418 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000359 | 359/1076 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000364 | 458/1060 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000468 | 378/1434 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000421 | 450/1253 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000424 | 459/1261 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000371 | 359/1117 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000405 | 386/1220 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000464 | 372/1423 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000473 | 350/1460 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000523 | 379/1617 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✕ERRORstop | code_repair | - | 0.000490 | 475/1474 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✕ERRORtimeout | code_repair | - | 0.000000 | 0/0 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000377 | 385/1127 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000258 | 454/709 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000382 | 374/1150 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✕ERRORstop | code_repair | - | 0.000497 | 366/1534 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000289 | 493/800 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000557 | 392/1726 | full transcript → |