Run camp-full_map-nano-002/breaking_point
91% PASSWhat this run revealed, then every individual trial as evidence.
56
10 errors
Trials
91%
42/46 scored
over trials
Pass rate
4
Failures
$0.0298
Run cost
Run camp-full_map-nano-002/breaking_point: 91% pass rate on belief queries.
Of 46 scored trials (belief query kind), the model passed 91% and failed 9%.
10 additional trial(s) ended in ERROR (infra failures, cost cap, or cancellation) and are excluded from the pass rate above.
4 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Diffuse weakness (below 90% but no outright flip) on: temporal (pass rate 89% at the high end).
In 2 of 4 classifiable belief failures the model reported where the object ACTUALLY was, not where the queried entity BELIEVED it was -- it collapsed their perspective into reality.
Notable failures
- record 93fd41ae expected: the basementactual: photo studio
- record 928137f8 expected: the basementactual: supply closet
- record 7d4fafff expected: the photo studioactual: library
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "garage" | belief | - | 0.000064 | 167/279 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "meditation room" | belief | - | 0.000095 | 176/430 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000133 | 174/624 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "east wing" | belief | - | 0.000157 | 178/739 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "east wing" | belief | - | 0.000066 | 163/288 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000085 | 171/381 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "atrium" | belief | - | 0.000094 | 164/430 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "conference room" | belief | - | 0.000611 | 514/2928 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "storage room" | belief | - | 0.000557 | 517/2657 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000826 | 521/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "workshop" | belief | - | 0.000416 | 493/1958 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "photo studio"unverified | belief | - | 0.000703 | 521/3385 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "supply closet"unverified | belief | - | 0.000363 | 496/1693 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "storage room" | belief | - | 0.000290 | 477/1332 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000289 | 870/1225 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "observation deck" | belief | - | 0.000554 | 869/2552 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "kitchen" | belief | - | 0.000204 | 837/809 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "garage" | belief | - | 0.000172 | 852/648 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "lobby" | belief | - | 0.000251 | 830/1049 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "archive room" | belief | - | 0.000543 | 835/2504 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "boiler room" | belief | - | 0.000534 | 855/2456 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000861 | 1225/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "gym" | belief | - | 0.000754 | 1211/3467 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "print room" | belief | - | 0.000784 | 1227/3615 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "reception area" | belief | - | 0.000439 | 1201/1894 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "archive room" | belief | - | 0.000297 | 1243/1173 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "wine cellar" | belief | - | 0.000702 | 1192/3210 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "gym" | belief | - | 0.000330 | 1188/1355 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000877 | 1533/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "archive room" | belief | - | 0.000660 | 1573/2907 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000875 | 1496/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000771 | 1540/3470 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "break room" | belief | - | 0.000306 | 1514/1150 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000834 | 1525/3789 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000501 | 1520/2127 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000853 | 1800/3815 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "workshop" | belief | - | 0.000464 | 1799/1868 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000889 | 1770/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "library"unverified | belief | - | 0.000790 | 1770/3508 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "wine cellar" | belief | - | 0.000543 | 1765/2273 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000793 | 1754/3524 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "server room" | belief | - | 0.000554 | 1796/2321 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000903 | 2054/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "loading dock" | belief | - | 0.000646 | 2110/2701 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "reception area"unverified | belief | - | 0.000372 | 2051/1349 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000904 | 2089/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000902 | 2039/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "archive room" | belief | - | 0.000541 | 2035/2194 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "conference room" | belief | - | 0.000783 | 2121/3387 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "reception area" | belief | - | 0.000412 | 2323/1479 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000445 | 2236/1666 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000527 | 2267/2066 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "server room" | belief | - | 0.000388 | 2274/1370 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000910 | 2201/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✕ERRORlength | belief | - | 0.000915 | 2291/4000 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000318 | 2308/1014 | full transcript → |