Run camp-determinism-probe-gemma4qat-001/parallel
27% PASSWhat this run revealed, then every individual trial as evidence.
15
Trials
27%
4/15 scored
over trials
Pass rate
11
Failures
$0.0088
Run cost
Run camp-determinism-probe-gemma4qat-001/parallel: 27% pass rate on code queries.
Of 15 scored trials (code query kind), the model passed 27% and failed 73%.
11 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 5f1d8e71 expected: (unknown)actual: ```
- record 9f95e142 expected: (unknown)actual: ```
- record 1b2672e1 expected: (unknown)actual: ```
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000361 | 459/1051 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000426 | 459/1268 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000405 | 386/1220 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000405 | 386/1220 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000426 | 459/1268 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000700 | 386/2206 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "```" | code_repair | - | 0.000306 | 364/898 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000313 | 364/921 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.000312 | 364/920 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.001892 | 386/6178 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```"unverified | code_repair | - | 0.001892 | 386/6178 | full transcript → |