Run camp-interactive-lookalike-super-001
85% PASSWhat this run revealed, then every individual trial as evidence.
25
5 errors
Trials
85%
17/20 scored
over trials
Pass rate
3
Failures
$0.2089
Run cost
Run camp-interactive-lookalike-super-001: 85% pass rate on interactive queries.
Of 20 scored trials (interactive query kind), the model passed 85% and failed 15%.
5 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.
3 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Sharp boundary detected on: final_distance (high-end pass rate 0%).
Diffuse weakness (below 90% but no outright flip) on: first_deviation_step (pass rate 71% at the high end), n_errors (pass rate 57% at the high end).
Notable failures
- record b0d02a0e expected: (unknown)actual: [{"step": 1, "tool": "set_-flag_0", "args": {"flag": "flag_0"}, "ok": true, "error_kind": null, "...
- record 10bca94a expected: (unknown)actual: [{"step": 1, "tool": "set_-flag_0", "args": {"flag": "flag_0"}, "ok": true, "error_kind": null, "...
- record 43d30d0e expected: (unknown)actual: [{"step": 1, "tool": "advance_o0", "args": {"service": "svc_0"}, "ok": true, "error_kind": null, ...
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.006257 | 56449/3869 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "build_art_0", "args": {"artifact": ..." | interactive | - | 0.006349 | 56413/4079 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.004878 | 48747/2173 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.004946 | 48887/2300 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..."unverified | interactive | - | 0.014489 | 110884/12484 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "adva_nce_0", "args": {"service": "s..." | interactive | - | 0.006822 | 68034/3066 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "build-_art_0", "args": {"artifact":..." | interactive | - | 0.006696 | 63650/3564 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.007314 | 68119/4144 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..."unverified | interactive | - | 0.003993 | 24451/4527 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.006708 | 63921/3544 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.009022 | 92520/3601 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.009823 | 97717/4458 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_-flag_0", "args": {"flag": "fla..." | interactive | - | 0.013807 | 130797/7430 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.009279 | 92506/4174 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.009108 | 92617/3775 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.011382 | 121770/3645 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.011624 | 121690/4198 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_o0", "args": {"flag": "fla..." | interactive | - | 0.014267 | 140627/6705 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "[{"step": 1, "tool": "set_flag_0", "args": {"flag": "flag..." | interactive | - | 0.011479 | 115902/4904 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "[{"step": 1, "tool": "advance_o0", "args": {"service": "s..."unverified | interactive | - | 0.026648 | 261908/12657 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕ERRORtool_calls | interactive | - | 0.005157 | 53060/2028 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕ERRORinfra failure | interactive | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕ERRORinfra failure | interactive | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕ERRORtool_calls | interactive | - | 0.004607 | 40905/2966 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕ERRORtool_calls | interactive | - | 0.004263 | 40905/2202 | full transcript → |