Run camp-nemrender-showcase-001
97% PASSWhat this run revealed, then every individual trial as evidence.
46
11 errors
Trials
97%
34/35 scored
over trials
Pass rate
1
Failures
$0.0079
Run cost
Run camp-nemrender-showcase-001: passed essentially everything (97%) -- this test did not challenge the model.
Of 35 scored trials (state query kind), the model passed 97%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 9b4d5c0c expected: the mail roomactual: loading dock
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "utility closet" | state | - | 0.000085 | 104/170 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | state | - | 0.000057 | 96/109 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000091 | 98/185 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000070 | 99/138 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | state | - | 0.000088 | 98/178 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | state | - | 0.000108 | 104/221 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "cafeteria" | state | - | 0.000153 | 236/299 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "utility closet" | state | - | 0.000084 | 110/167 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000087 | 105/175 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000101 | 104/207 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | state | - | 0.000077 | 102/152 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "boiler room" | state | - | 0.000170 | 244/335 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "boiler room" | state | - | 0.000312 | 2031/333 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "cafeteria" | state | - | 0.000130 | 216/251 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "boiler room" | state | - | 0.000163 | 237/321 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "observation deck" | state | - | 0.000172 | 247/339 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "east wing" | state | - | 0.000280 | 394/552 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000182 | 257/359 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "observation deck" | state | - | 0.000431 | 2031/597 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "south stairwell" | state | - | 0.000255 | 382/499 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "east wing" | state | - | 0.000318 | 2029/347 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "mail room" | state | - | 0.000259 | 397/504 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "loading dock" | state | - | 0.000246 | 384/478 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | state | - | 0.000309 | 504/596 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "atrium" | state | - | 0.000311 | 518/600 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | state | - | 0.000265 | 2031/227 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "loading dock"unverified | state | - | 0.000313 | 536/601 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "west wing" | state | - | 0.000438 | 2032/613 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "atrium" | state | - | 0.000251 | 2032/196 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000391 | 631/756 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "print room" | state | - | 0.000269 | 512/507 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000322 | 2030/354 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "rooftop terrace" | state | - | 0.000368 | 665/699 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "mail room" | state | - | 0.000278 | 2029/256 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/Nemotron-3_5-Lightning | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?REJECTED_BY_FIDELITY_GATEinfra failure | - | - | 0.000000 | 0/0 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000466 | 2029/674 | full transcript → |