Data to …… trials

Run camp-interactive-lookalike-nano-002

67% PASS

What this run revealed, then every individual trial as evidence.

watch live →compare across models →
15
Trials
67%
10/15 scored
over trials
Pass rate
5
33%
Failures
$0.0608
Run cost

Run camp-interactive-lookalike-nano-002: 67% pass rate on interactive queries.

Of 15 scored trials (interactive query kind), the model passed 67% and failed 33%.

5 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Sharp boundary detected on: final_distance (high-end pass rate 0%).

Notable failures

  • record adc60a4d — asked "You may make at most 18 tool calls."
    expected: (unknown)
    actual: [{"step": 1, "tool": "build-_art_0", "args": {"artifact": "art_0"}, "ok": true, "error_kind": nul...

    answer did not match the verified ground truth

  • record 4760dc52 — asked "You may make at most 18 tool calls."
    expected: (unknown)
    actual: [{"step": 1, "tool": "adva-nce_0", "args": {"service": "svc_0"}, "ok": true, "error_kind": null, ...

    answer did not match the verified ground truth

  • record c73eb1a5 — asked "You may make at most 18 tool calls."
    expected: (unknown)
    actual: [{"step": 1, "tool": "build_art_0", "args": {"artifact": "art_0"}, "ok": true, "error_kind": null...

    answer did not match the verified ground truth

Trials15 trials
Per-trial table (15 trials -- prompt, reasoning, verdict, answer key)
modelverdictkinddifficultycost ($)tokens in/out
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "[{"step": 1, "tool": "build-_art_0", "args": {"artifact":..."unverifiedinteractive-0.00345015080/13478full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..."interactive-0.00481919507/19218full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..."interactive-0.00408919516/15564full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "[{"step": 1, "tool": "adva-nce_0", "args": {"service": "s..."unverifiedinteractive-0.0022397106/9416full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build_art_o0", "args": {"artifact":..."interactive-0.00348519372/12582full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..."interactive-0.00584321772/23771full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00772121639/33193full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..."interactive-0.00560021762/22560full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "build-_art_0", "args": {"artifact":..."interactive-0.00419919441/16133full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..."unverifiedinteractive-0.00460819490/18166full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..."interactive-0.00310017140/11215full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "[]"unverifiedinteractive-0.0011791700/5469full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B?FAILanswered "[]"unverifiedinteractive-0.0007821698/3487full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..."interactive-0.00505321767/19823full transcript →
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..."interactive-0.00460217196/18709full transcript →