Run camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001
98% PASSWhat this run revealed, then every individual trial as evidence.
40
Trials
98%
39/40 scored
over trials
Pass rate
1
Failures
$0.0057
Run cost
Run camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001: passed essentially everything (98%) -- this test did not challenge the model.
Of 40 scored trials (tool query kind), the model passed 98%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 78829e71 expected: (unknown)actual: [{"tool":"provision","args":{"service":"svc_0"}}]
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000072 | 387/264 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000087 | 387/339 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000090 | 384/355 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000090 | 387/354 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000123 | 385/518 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000166 | 434/721 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000126 | 433/524 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000122 | 431/504 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000144 | 435/611 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000149 | 431/637 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000213 | 525/935 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000209 | 524/915 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000182 | 525/779 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000192 | 526/829 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000239 | 526/1065 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000227 | 525/1003 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000158 | 527/659 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000203 | 525/886 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000209 | 523/914 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000175 | 527/741 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000224 | 522/990 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000155 | 526/643 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..." | tool_plan | - | 0.000248 | 525/1111 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000131 | 526/524 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "]" | tool_plan | - | 0.000197 | 527/853 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000076 | 387/281 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000086 | 386/333 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000088 | 384/343 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000092 | 384/365 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "[{"tool":"provision","args":{"service":"svc_0"}}]"unverified | tool_plan | - | 0.000103 | 425/410 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000120 | 383/502 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000076 | 425/273 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000095 | 426/366 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000103 | 424/407 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000115 | 428/469 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000118 | 469/475 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000116 | 471/461 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000087 | 472/318 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000119 | 471/476 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]" | tool_plan | - | 0.000128 | 469/525 | full transcript → |