Run camp-consistency-gemma4qat-001
47% PASSWhat this run revealed, then every individual trial as evidence.
20
1 error
Trials
47%
9/19 scored
over trials
Pass rate
10
Failures
$0.0047
Run cost
Run camp-consistency-gemma4qat-001: 47% pass rate on tool queries.
Of 19 scored trials (tool query kind), the model passed 47% and failed 53%.
1 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.
10 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 552de0d5 expected: (unknown)actual: ]
- record d38ee364 expected: (unknown)actual: ```
- record 080612c5 expected: (unknown)actual: ```
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000248 | 1337/382 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000248 | 1337/382 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000248 | 1337/382 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✕ERRORtimeout | tool_plan | - | 0.000000 | 0/0 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000248 | 1337/382 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1341/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000251 | 1341/389 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "```" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000251 | 1341/389 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1341/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1341/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000249 | 1338/383 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000249 | 1338/383 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000249 | 1338/383 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000249 | 1338/383 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000249 | 1338/383 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1340/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1340/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1340/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1340/384 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000249 | 1340/384 | full transcript → |