Run camp-irrelevance-super-001
93% PASSWhat this run revealed, then every individual trial as evidence.
30
Trials
93%
28/30 scored
over trials
Pass rate
2
Failures
$0.0120
Run cost
Run camp-irrelevance-super-001: 93% pass rate on tool queries.
Of 30 scored trials (tool query kind), the model passed 93% and failed 7%.
2 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Diffuse weakness (below 90% but no outright flip) on: unsolvable (pass rate 83% at the high end).
Notable failures
- record 6049f247 expected: (unknown)actual: [{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":"set_flag_0","args":{"flag":"flag_0"}},{...
- record 361d3e3c expected: (unknown)actual: ]
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000246 | 774/410 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000243 | 773/403 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000237 | 779/389 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000409 | 823/762 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000524 | 825/1017 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000504 | 827/972 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}, {"t..." | tool_plan | - | 0.000426 | 826/799 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000447 | 828/846 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000217 | 789/343 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000539 | 824/1051 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000433 | 826/816 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000666 | 826/1333 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000211 | 774/332 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000313 | 774/557 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000334 | 778/604 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000253 | 789/422 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."unverified | tool_plan | - | 0.000381 | 780/708 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "]"unverified | tool_plan | - | 0.000471 | 775/909 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000509 | 824/985 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000395 | 825/732 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000618 | 827/1226 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000484 | 827/929 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000420 | 826/787 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000415 | 825/775 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000540 | 825/1054 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000195 | 788/294 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000399 | 823/740 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[]" | tool_plan | - | 0.000175 | 787/249 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000521 | 826/1012 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "]" | tool_plan | - | 0.000507 | 826/980 | full transcript → |