Run camp-ef-tickets-gemma4qat-001/C
50% PASSWhat this run revealed, then every individual trial as evidence.
10
Trials
50%
5/10 scored
over trials
Pass rate
5
Failures
$0.0074
Run cost
Run camp-ef-tickets-gemma4qat-001/C: 50% pass rate on interactive queries.
Of 10 scored trials (interactive query kind), the model passed 50% and failed 50%.
5 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Sharp boundary detected on: final_distance (high-end pass rate 0%).
Notable failures
- record d56f2023 expected: (unknown)actual: [{"step": 1, "tool": "advance_0", "args": {"service": "svc_0"}, "ok": true, "error_kind": null, "...
- record 64ae0031 expected: (unknown)actual: [{"step": 1, "tool": "advance_0", "args": {"service": "svc_0"}, "ok": true, "error_kind": null, "...
- record b216893a expected: (unknown)actual: [{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag_0"}, "ok": true, "error_kind": null, "r...
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."unverified | interactive | - | 0.000735 | 5588/587 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..." | interactive | - | 0.000747 | 5497/657 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..." | interactive | - | 0.000700 | 5410/529 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..." | interactive | - | 0.000826 | 5603/885 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."unverified | interactive | - | 0.000681 | 5496/437 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."unverified | interactive | - | 0.000722 | 5544/557 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."unverified | interactive | - | 0.000744 | 5614/609 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..." | interactive | - | 0.000746 | 5550/635 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ?FAILanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."unverified | interactive | - | 0.000731 | 5505/601 | full transcript → |
| gemma-4-31b-it-qat-w4a16-ct | ✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..." | interactive | - | 0.000735 | 5531/605 | full transcript → |