Run camp-determinism-probe-gemma4qat-001/serial
33% PASSWhat this run revealed, then every individual trial as evidence.
15
Trials
33%
5/15 scored
over trials
Pass rate
10
Failures
$0.0100
Run cost
Run camp-determinism-probe-gemma4qat-001/serial: 33% pass rate on code queries.
Of 15 scored trials (code query kind), the model passed 33% and failed 67%.
10 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 7b2c42df expected: (unknown)actual: ```
- record dcc2d5eb expected: (unknown)actual: ```
- record 2f53b5db expected: (unknown)actual: ```
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000561 | 459/1717 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000327 | 386/962 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.001436 | 386/4658 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.001436 | 386/4658 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.001436 | 386/4658 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.001436 | 386/4658 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000644 | 364/2025 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |