Run camp-full_map-super-002/state
93% PASSWhat this run revealed, then every individual trial as evidence.
30
Trials
97%
28/29 scored
over trials
Pass rate
2
1 stable / 1 noise
Failures
$0.0133
Run cost
Run camp-full_map-super-002/state: 97% pass rate on state queries.
Of 29 scored trials (state query kind), the model passed 97% and failed 3%.
2 failures recorded; 1 survived verification (1 were sampling noise and are excluded from findings).
Notable failures
- record bdc7a710 expected: the atriumactual: wine cellar
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | state | - | 0.000063 | 128/118 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "utility closet" | state | - | 0.000085 | 140/164 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000114 | 131/231 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | state | - | 0.000128 | 134/260 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "boiler room" | state | - | 0.000157 | 265/302 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000231 | 133/489 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "cafeteria" | state | - | 0.000214 | 256/429 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "boiler room" | state | - | 0.000201 | 255/402 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "observation deck" | state | - | 0.000349 | 268/729 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000325 | 272/674 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "loading dock" | state | - | 0.000451 | 383/934 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "east wing" | state | - | 0.000310 | 406/616 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "cafeteria" | state | - | 0.000472 | 405/978 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕FAILanswered "photo studio"noise | state | - | 0.000243 | 398/469 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "mail room" | state | - | 0.000462 | 401/956 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "Archive Room" | state | - | 0.000468 | 504/951 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "west wing" | state | - | 0.000453 | 525/914 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✕FAILanswered "wine cellar"verified-stable | state | - | 0.000585 | 519/1207 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "print room" | state | - | 0.000457 | 519/923 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "mail room" | state | - | 0.000604 | 518/1250 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "rooftop terrace" | state | - | 0.000453 | 663/888 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000523 | 657/1045 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "east wing" | state | - | 0.000547 | 649/1101 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | state | - | 0.000799 | 641/1662 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000697 | 629/1437 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "print room" | state | - | 0.000506 | 657/1008 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | state | - | 0.000555 | 682/1113 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "supply closet" | state | - | 0.000676 | 667/1383 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "south stairwell" | state | - | 0.000676 | 625/1391 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "cafeteria" | state | - | 0.001466 | 679/3138 | full transcript → |