Data to …… trials

Run camp-propose-toolworld-001/control

79% PASS

What this run revealed, then every individual trial as evidence.

20
6 errors
Trials
79%
11/14 scored
over trials
Pass rate
3
15%
Failures
$0.0105
Run cost

Run camp-propose-toolworld-001/control: 79% pass rate on tool queries.

Of 14 scored trials (tool query kind), the model passed 79% and failed 21%.

6 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.

3 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Notable failures

  • record 79dac819 — asked "You may use at most 20 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

  • record 3628cbcd — asked "You may use at most 6 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

  • record 82ccae20 — asked "You may use at most 6 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

Trials20 trials
Per-trial table (20 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000598884/2770full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000769893/3620full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000673897/3139full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000440892/1978full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000636889/2957full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000841827/4000full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000842831/4000full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000841824/4000full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000841819/4000full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000843852/4000full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000314964/1330full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000348950/1500full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000323962/1376full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000334955/1433full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000316953/1342full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000320809/1400full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000304816/1316full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000340807/1500full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000277805/1186full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000276806/1180full transcript →