Data to …… trials

Run camp-tool_bp-super-001

98% PASS

What this run revealed, then every individual trial as evidence.

watch live →compare across models →
56
4 errors
Trials
98%
51/52 scored
over trials
Pass rate
1
2%
Failures
$0.0348
Run cost

Run camp-tool_bp-super-001: passed essentially everything (98%) -- this test did not challenge the model.

Of 52 scored trials (tool query kind), the model passed 98%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.

4 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.

1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Notable failures

  • record 180b873c — asked "You may use at most 12 tool calls."
    expected: (unknown)
    actual: ]

    answer did not match the verified ground truth

Trials56 trials
Per-trial table (56 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000179564/298full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000186561/313full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}, {"t..."tool_plan-0.000167564/271full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000160561/255full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000151563/235full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000152560/238full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000195564/334full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000184563/309full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000160563/256full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool": "advance_0", "args": {"service": "svc_0"}}, {"t..."tool_plan-0.000194564/331full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000197564/338full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000225564/400full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000179562/297full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000217562/383full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000433727/834full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000487726/954full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000343727/632full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000448726/867full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000452724/875full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000487726/954full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000349723/648full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":..."tool_plan-0.000499725/980full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000374726/703full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000446728/861full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000511728/1007full transcript →
nvidia/nemotron-3-super-120b-a12b?FAILanswered "]" (world seeded for: poison_traps=0)unverifiedtool_plan-0.000337728/620full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000378725/712full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000399726/757full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000716954/1422full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000895965/1817full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.001031958/2121full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000702960/1390full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000874967/1770full transcript →
nvidia/nemotron-3-super-120b-a12b✕ERRORlengthtool_plan-0.000752960/1500full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.000837955/1691full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0007181040/1411full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0013271037/2764full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0009131047/1842full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0010591044/2167full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0013591039/2835full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0013721046/2862full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0009481035/1922full transcript →
nvidia/nemotron-3-super-120b-a12b✕ERRORlengthtool_plan-0.0007581035/1500full transcript →
nvidia/nemotron-3-super-120b-a12b✕ERRORlengthtool_plan-0.0007591045/1500full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0012471035/2587full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0009811043/1994full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0011481034/2368full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0009811043/1995full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0009611040/1951full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0008721039/1752full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0006961041/1362full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0010481040/2143full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0007371033/1454full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0008391036/1680full transcript →
nvidia/nemotron-3-super-120b-a12b✕ERRORlengthtool_plan-0.0007581042/1500full transcript →
nvidia/nemotron-3-super-120b-a12b✓PASSanswered "]"tool_plan-0.0010151034/2071full transcript →