Data to …… trials

Run camp-tool-deep-nano-001

29% PASS

What this run revealed, then every individual trial as evidence.

7
Trials
29%
2/7 scored
over trials
Pass rate
5
71%
Failures
$0.0029
Run cost

Run camp-tool-deep-nano-001: 29% pass rate on tool queries.

Of 7 scored trials (tool query kind), the model passed 29% and failed 71%.

5 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Notable failures

  • record 5d827849 — asked "You may use at most 14 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

  • record 12276700 — asked "You may use at most 14 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

  • record 0a5587f3 — asked "You may use at most 14 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

Trials7 trials
Per-trial table (7 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000270783/1156full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: id_collisions_on=1)unverifiedtool_plan-0.000378783/1692full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: id_collisions_on=1)unverifiedtool_plan-0.000440786/2005full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: id_collisions_on=1)unverifiedtool_plan-0.000454784/2075full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: id_collisions_on=1)unverifiedtool_plan-0.000462788/2111full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "]"tool_plan-0.000596787/2782full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "]" (world seeded for: id_collisions_on=1)unverifiedtool_plan-0.000349784/1548full transcript →