Run camp-belief-nemotron-nano-001
93% PASSWhat this run revealed, then every individual trial as evidence.
105
Trials
93%
98/105 scored
over trials
Pass rate
7
Failures
$0.0139
Run cost
Run camp-belief-nemotron-nano-001: 93% pass rate on belief queries.
Of 105 scored trials (belief query kind), the model passed 93% and failed 7%.
7 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Diffuse weakness (below 90% but no outright flip) on: entity_state (pass rate 84% at the high end).
7 belief FAIL(s) were recorded, but they predate the generator-hardening fix and can't be safely re-classified against regenerated ground truth.
Notable failures
- record 464c525b expected: (unknown)actual: loading dock
- record d3b6c6ce expected: (unknown)actual: archive room
- record 3ef94581 expected: (unknown)actual: lobby
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000168 | 158/802 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "storage room" | belief | - | 0.000096 | 165/441 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "break room" | belief | - | 0.000128 | 143/604 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "kitchen" | belief | - | 0.000034 | 140/137 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "photo studio" | belief | - | 0.000033 | 152/127 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "storage room" | belief | - | 0.000074 | 225/314 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "loading dock"unverified | belief | - | 0.000089 | 223/388 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000234 | 226/1112 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000113 | 224/509 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "library" | belief | - | 0.000071 | 192/305 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "archive room"unverified | belief | - | 0.000181 | 280/835 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "reception area" | belief | - | 0.000218 | 237/1033 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "south stairwell" | belief | - | 0.000276 | 281/1309 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "lobby"unverified | belief | - | 0.000202 | 268/944 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000215 | 281/1007 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "atrium" | belief | - | 0.000198 | 323/911 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "archive room" | belief | - | 0.000199 | 311/917 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000124 | 337/538 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "courtyard" | belief | - | 0.000174 | 327/790 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "east wing" | belief | - | 0.000245 | 347/1136 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "mail room" | belief | - | 0.000635 | 389/3078 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000268 | 396/1243 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "library"unverified | belief | - | 0.000236 | 409/1080 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "basement" | belief | - | 0.000201 | 401/906 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "south stairwell" | belief | - | 0.000253 | 399/1167 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "kitchen" | belief | - | 0.000072 | 153/323 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000090 | 169/409 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000063 | 145/279 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000067 | 163/295 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "server room" | belief | - | 0.000162 | 165/768 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "workshop" | belief | - | 0.000087 | 158/397 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "gym" | belief | - | 0.000136 | 155/639 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000073 | 150/330 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000123 | 160/577 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000238 | 161/1151 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "lobby" | belief | - | 0.000154 | 157/732 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000125 | 164/583 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "photo studio" | belief | - | 0.000232 | 161/1122 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "kitchen" | belief | - | 0.000067 | 162/293 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000123 | 148/579 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000123 | 145/580 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000140 | 158/661 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "storage room" | belief | - | 0.000046 | 165/188 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "photo studio" | belief | - | 0.000070 | 165/307 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "reception area" | belief | - | 0.000037 | 165/143 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "garage" | belief | - | 0.000085 | 140/388 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "north stairwell" | belief | - | 0.000201 | 144/967 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "west wing"unverified | belief | - | 0.000227 | 162/1092 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "north stairwell" | belief | - | 0.000072 | 144/324 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000094 | 148/432 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "observation deck" | belief | - | 0.000031 | 143/121 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "gym" | belief | - | 0.000088 | 148/405 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "lobby" | belief | - | 0.000044 | 146/182 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000129 | 162/603 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "observation deck"unverified | belief | - | 0.000150 | 158/708 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "east wing" | belief | - | 0.000062 | 156/273 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000164 | 175/775 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ?FAILanswered "security office"unverified | belief | - | 0.000289 | 174/1402 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "garage" | belief | - | 0.000118 | 167/549 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000192 | 176/915 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "meditation room" | belief | - | 0.000073 | 182/319 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "mail room" | belief | - | 0.000065 | 180/280 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "reception area" | belief | - | 0.000062 | 183/264 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "reception area" | belief | - | 0.000088 | 185/395 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "mail room" | belief | - | 0.000054 | 176/227 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "north stairwell" | belief | - | 0.000067 | 218/282 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "basement" | belief | - | 0.000094 | 213/415 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "archive room" | belief | - | 0.000098 | 204/440 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000064 | 211/265 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "north stairwell" | belief | - | 0.000065 | 223/267 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "garage" | belief | - | 0.000076 | 225/326 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "lobby" | belief | - | 0.000142 | 233/651 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "courtyard" | belief | - | 0.000057 | 229/227 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "library" | belief | - | 0.000076 | 228/325 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "supply closet" | belief | - | 0.000084 | 224/363 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "courtyard" | belief | - | 0.000065 | 162/285 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000087 | 160/397 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "south stairwell" | belief | - | 0.000053 | 173/221 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000064 | 153/281 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000086 | 159/390 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "courtyard" | belief | - | 0.000105 | 232/466 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "workshop" | belief | - | 0.000047 | 232/179 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "courtyard" | belief | - | 0.000348 | 216/1684 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "cafeteria" | belief | - | 0.000063 | 218/260 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "meditation room" | belief | - | 0.000062 | 221/253 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "gym" | belief | - | 0.000087 | 312/359 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "east wing" | belief | - | 0.000150 | 310/673 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000047 | 316/155 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "cafeteria" | belief | - | 0.000121 | 342/517 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "workshop" | belief | - | 0.000126 | 327/548 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "break room" | belief | - | 0.000101 | 423/400 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000190 | 427/844 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "basement" | belief | - | 0.000049 | 380/151 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000058 | 432/180 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "west wing" | belief | - | 0.000125 | 400/524 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "kitchen" | belief | - | 0.000080 | 470/284 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000079 | 498/271 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000056 | 492/159 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "boiler room" | belief | - | 0.000224 | 479/998 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "rooftop terrace" | belief | - | 0.000065 | 490/204 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "security office" | belief | - | 0.000260 | 823/1094 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "greenhouse" | belief | - | 0.000248 | 797/1041 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "workshop" | belief | - | 0.000218 | 766/897 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "utility closet" | belief | - | 0.000300 | 834/1294 | full transcript → |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B | ✓PASSanswered "front office" | belief | - | 0.000324 | 785/1426 | full transcript → |