Data to …… trials

Run camp-interactive-nano-001

97% PASS

What this run revealed, then every individual trial as evidence.

watch live →compare across models →
40
3 errors
Trials
97%
36/37 scored
over trials
Pass rate
1
2%
Failures
$0.3137
Run cost

Run camp-interactive-nano-001: passed essentially everything (97%) -- this test did not challenge the model.

Of 37 scored trials (interactive query kind), the model passed 97%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.

3 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.

1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Sharp boundary detected on: final_distance (high-end pass rate 0%).

Notable failures

  • record 26c3dca1 — asked "You may make at most 32 tool calls."
    expected: (unknown)
    actual: [{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag_0"}, "ok": true, "error_kind": null, "r...

    answer did not match the verified ground truth

Trials40 trials
Per-trial table (40 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00142810968/4400full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00171010951/5811full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00179410958/6228full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00150510954/4788full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00194212777/6515full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00341817875/12621full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00369817828/14034full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00341317827/12608full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00279117745/9517full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00403817772/15747full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00519426644/19307full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00582126898/22380full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00621726890/24363full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00501326806/18362full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00582026691/22427full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00712733747/27200full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00817133956/32365full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00718833592/27540full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00740333760/28577full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00864733840/34773full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.00915548893/33553full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.00917848972/33649full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00965449076/36003full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.01057248641/40698full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01164348926/45985full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."unverifiedinteractive-0.00763436019/29165full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01226472861/43104full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01503667198/58381full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.01204567142/43441full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01801667688/73160full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.01056491584/29923full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01377592142/45838full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01444791691/49314full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.01208192345/37320full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "advance_0", "args": {"service": "sv..."interactive-0.01214691676/37813full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."interactive-0.012413105234/35757full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.014682111773/45469full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORtool_callsinteractive-0.01325880930/46057full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORinfra failureinteractive-0.0000000/0full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✕ERRORtool_callsinteractive-0.00275014782/10053full transcript →