Data to …… trials

Run camp-propose-toolworld-001/round3

79% PASS

What this run revealed, then every individual trial as evidence.

20
1 error
Trials
79%
15/19 scored
over trials
Pass rate
4
20%
Failures
$0.0090
Run cost

Run camp-propose-toolworld-001/round3: 79% pass rate on tool queries.

Of 19 scored trials (tool query kind), the model passed 79% and failed 21%.

1 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.

4 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Notable failures

  • record afad433b — asked "You may use at most 8 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

  • record 5218e1a5 — asked "You may use at most 14 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

  • record 02ba1d1a — asked "You may use at most 14 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

Trials20 trials
Per-trial table (20 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0006071045/2772full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"tool":"build_art_0","args":{"artifact":"art_0"}},{"too..."tool_plan-0.0007341053/3406full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0005831050/2653full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0004561040/2019full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0005691047/2583full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"tool":"advance_o0","args":{"service":"svc_0"}},{"tool"..."tool_plan-0.000200507/875full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000157521/653full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000263510/1186full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000208508/914full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000154515/639full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0005561069/2515full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0008311054/3890full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0003901053/1688full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0007071067/3270full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.0005661073/2564full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000592813/2756full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000459814/2093full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORlengthtool_plan-0.000341823/1500full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000279810/1193full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000313807/1361full transcript →