Run camp-full_map-ultra-002/state
PASSEDWhat this run revealed, then every individual trial as evidence.
30
Trials
100%
30/30 scored
over trials
Pass rate
0
Failures
$0.0431
Run cost
Run camp-full_map-ultra-002/state: passed essentially everything (100%) -- this test did not challenge the model.
Of 30 scored trials (state query kind), the model passed 100%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "storage room" | state | - | 0.000493 | 133/194 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "utility closet" | state | - | 0.000497 | 140/194 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "courtyard" | state | - | 0.000529 | 134/210 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "courtyard" | state | - | 0.000511 | 128/203 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "photo studio" | state | - | 0.000512 | 131/203 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "boiler room" | state | - | 0.000746 | 265/279 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "cafeteria" | state | - | 0.000691 | 256/256 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "boiler room" | state | - | 0.000911 | 255/356 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "storage room" | state | - | 0.001023 | 272/403 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "loading dock" | state | - | 0.001157 | 383/439 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "observation deck" | state | - | 0.001091 | 268/435 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "east wing" | state | - | 0.001497 | 406/588 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "Cafeteria" | state | - | 0.001091 | 405/404 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "south stairwell" | state | - | 0.001028 | 398/377 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "Archive Room" | state | - | 0.001326 | 504/488 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "mail room" | state | - | 0.001617 | 401/644 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "west wing" | state | - | 0.001565 | 525/592 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "atrium" | state | - | 0.001745 | 519/675 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "photo studio" | state | - | 0.001648 | 629/606 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "mail room" | state | - | 0.001700 | 518/655 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "photo studio" | state | - | 0.001781 | 657/660 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "storage room" | state | - | 0.001889 | 641/713 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "print room" | state | - | 0.001753 | 519/679 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "rooftop terrace" | state | - | 0.002558 | 663/1012 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "east wing" | state | - | 0.002034 | 649/777 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "photo studio" | state | - | 0.001967 | 682/739 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "cafeteria" | state | - | 0.001926 | 679/721 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "south stairwell" | state | - | 0.001879 | 625/712 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "print room" | state | - | 0.002649 | 657/1055 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "supply closet" | state | - | 0.003246 | 667/1324 | full transcript → |