Run camp-ef-tools-gemma4qat-001/B
80% PASSWhat this run revealed, then every individual trial as evidence.
10
Trials
80%
8/10 scored
over trials
Pass rate
2
Failures
$0.0023
Run cost
Run camp-ef-tools-gemma4qat-001/B: 80% pass rate on tool queries.
Of 10 scored trials (tool query kind), the model passed 80% and failed 20%.
2 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 7f694fdc expected: (unknown)actual: ]
- record 2aa4fd9f expected: (unknown)actual: ]
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000231 | 1260/351 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000232 | 1262/352 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000233 | 1264/354 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "]" (world seeded for: id_collisions_on=1)unverified | tool_plan | - | 0.000232 | 1262/353 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000231 | 1260/350 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000232 | 1262/352 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000233 | 1263/355 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000231 | 1259/350 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000233 | 1267/355 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "]" | tool_plan | - | 0.000231 | 1261/351 | full transcript → |