Run camp-interactive-lookalike-super-002
93% PASSWhat this run revealed, then every individual trial as evidence.
15
1 error
Trials
93%
13/14 scored
over trials
Pass rate
1
Failures
$0.2527
Run cost
Run camp-interactive-lookalike-super-002: 93% pass rate on interactive queries.
Of 14 scored trials (interactive query kind), the model passed 93% and failed 7%.
1 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.
1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Sharp boundary detected on: final_distance (high-end pass rate 0%).
Diffuse weakness (below 90% but no outright flip) on: n_calls (pass rate 86% at the high end), n_errors (pass rate 83% at the high end).
Notable failures
- record 53943bc6 expected: (unknown)actual: [{"step": 1, "tool": "set_flag_o0", "args": {"flag": "flag_0"}, "ok": true, "error_kind": null, "...
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "adva-nce_0", "args": {"service": "s..." | interactive | - | 0.012440 | 135287/3593 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.012888 | 135869/4485 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "build_art_o0", "args": {"artifact":..." | interactive | - | 0.011820 | 129099/3315 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "advance_o0", "args": {"service": "s..." | interactive | - | 0.013043 | 135400/4913 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..."unverified | interactive | - | 0.036049 | 296122/27466 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.014213 | 149558/4997 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..." | interactive | - | 0.017309 | 163415/9412 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.020431 | 163850/16273 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.014007 | 143122/5683 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.020366 | 192818/10978 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.016399 | 165476/7024 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.016665 | 172968/6284 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.018212 | 180234/8429 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.016769 | 179791/5302 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕ERRORtool_calls | interactive | - | 0.012053 | 110963/7057 | full transcript → |