Run camp-tool_planning-adv-ultra-001
PASSEDWhat this run revealed, then every individual trial as evidence.
40
Trials
100%
40/40 scored
over trials
Pass rate
0
Failures
$0.0373
Run cost
Run camp-tool_planning-adv-ultra-001: passed essentially everything (100%) -- this test did not challenge the model.
Of 40 scored trials (tool query kind), the model passed 100%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000673 | 386/218 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000668 | 385/216 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000739 | 386/248 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000816 | 386/283 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000816 | 387/283 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001258 | 434/473 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001094 | 432/399 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001072 | 433/389 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001156 | 434/427 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001173 | 431/435 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001583 | 526/600 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000762 | 525/227 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000846 | 525/265 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000899 | 527/289 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000971 | 525/322 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001402 | 524/518 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001514 | 525/569 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000805 | 523/247 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000837 | 522/262 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000850 | 525/267 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000907 | 525/293 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001202 | 525/427 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001404 | 525/519 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000816 | 524/252 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.001534 | 524/578 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000519 | 387/148 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000812 | 427/272 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000746 | 387/251 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000762 | 385/259 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000764 | 384/260 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000829 | 387/289 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000729 | 428/234 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000676 | 428/210 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000669 | 472/197 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "]" | tool_plan | - | 0.000739 | 427/239 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000774 | 426/255 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.001158 | 472/419 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000719 | 470/220 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000748 | 471/233 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}]" | tool_plan | - | 0.000893 | 470/299 | full transcript → |