Run camp-full_map-super-002/breaking_point
96% PASSWhat this run revealed, then every individual trial as evidence.
56
Trials
96%
54/56 scored
over trials
Pass rate
2
Failures
$0.0448
Run cost
Run camp-full_map-super-002/breaking_point: 96% pass rate on belief queries.
Of 56 scored trials (belief query kind), the model passed 96% and failed 4%.
2 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
In 1 of 2 classifiable belief failures the model's wrong answer matches the 'most_frequent_location' shortcut -- a plausible but ungrounded guess rather than tracking what that entity actually witnessed.
Notable failures
- record 26f5b058 expected: the basementactual: loading dock
- record 1b2f79d9 expected: the wine cellaractual: reception area
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "garage" | belief | - | 0.000155 | 167/315 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "meditation room" | belief | - | 0.000168 | 176/342 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "rooftop terrace" | belief | - | 0.000264 | 174/555 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "east wing" | belief | - | 0.000240 | 178/502 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "east wing" | belief | - | 0.000158 | 163/322 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "utility closet" | belief | - | 0.000137 | 171/274 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "atrium" | belief | - | 0.000163 | 164/334 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "conference room" | belief | - | 0.000978 | 514/2083 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | belief | - | 0.001048 | 517/2236 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "utility closet" | belief | - | 0.000348 | 521/681 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "workshop" | belief | - | 0.000896 | 493/1904 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "loading dock"unverified | belief | - | 0.000726 | 521/1520 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "basement" | belief | - | 0.001382 | 496/2984 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "storage room" | belief | - | 0.000488 | 477/999 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "rooftop terrace" | belief | - | 0.000588 | 870/1151 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "observation deck" | belief | - | 0.000568 | 869/1107 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "kitchen" | belief | - | 0.000683 | 837/1369 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "garage" | belief | - | 0.000702 | 852/1409 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "lobby" | belief | - | 0.000785 | 830/1597 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | belief | - | 0.000519 | 835/1004 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "boiler room" | belief | - | 0.000575 | 855/1125 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "front office" | belief | - | 0.000750 | 1225/1449 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "gym" | belief | - | 0.000547 | 1211/1000 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "print room" | belief | - | 0.000706 | 1227/1350 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "reception area" | belief | - | 0.000551 | 1201/1012 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | belief | - | 0.000513 | 1243/920 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "wine cellar" | belief | - | 0.001039 | 1192/2097 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "gym" | belief | - | 0.000663 | 1188/1263 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "supply closet" | belief | - | 0.000958 | 1533/1856 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | belief | - | 0.000702 | 1573/1280 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | belief | - | 0.000666 | 1496/1215 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "greenhouse" | belief | - | 0.000647 | 1540/1164 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "break room" | belief | - | 0.001381 | 1514/2800 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "rooftop terrace" | belief | - | 0.000658 | 1525/1192 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "west wing" | belief | - | 0.000874 | 1520/1671 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "rooftop terrace" | belief | - | 0.001744 | 1800/3555 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "workshop" | belief | - | 0.001902 | 1799/3906 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "courtyard" | belief | - | 0.000799 | 1770/1461 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "photo studio" | belief | - | 0.000962 | 1770/1822 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ?FAILanswered "reception area"unverified | belief | - | 0.001559 | 1765/3150 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "greenhouse" | belief | - | 0.000705 | 1754/1255 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "server room" | belief | - | 0.000817 | 1796/1497 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "wine cellar" | belief | - | 0.001038 | 2054/1942 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "loading dock" | belief | - | 0.000729 | 2110/1245 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "meditation room" | belief | - | 0.000782 | 2051/1374 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "south stairwell" | belief | - | 0.000594 | 2089/948 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "workshop" | belief | - | 0.000995 | 2039/1848 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "archive room" | belief | - | 0.000708 | 2035/1212 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "conference room" | belief | - | 0.001723 | 2121/3452 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "reception area" | belief | - | 0.001370 | 2323/2632 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "supply closet" | belief | - | 0.001284 | 2236/2456 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "supply closet" | belief | - | 0.001467 | 2267/2857 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "server room" | belief | - | 0.000829 | 2274/1437 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "server room" | belief | - | 0.000663 | 2201/1083 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "wine cellar" | belief | - | 0.001072 | 2291/1976 | full transcript → |
| nvidia/nemotron-3-super-120b-a12b | ✓PASSanswered "front office" | belief | - | 0.000819 | 2308/1410 | full transcript → |