Trials · state
Every trial behind one square of the capability map: Nemotron 3 Ultra 550B on state.
30
0 errors
Trials in this cell
100%
30/30 scored
Pass rate
0
0 noise excluded
Real failures
$0.0431
Cell cost
Trials
| kind | verdict | realized dims | seeded traps | run | |
|---|---|---|---|---|---|
| state | ✓PASSanswered "storage room" | entity_state=2 temporal=0 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "utility closet" | entity_state=2 temporal=0 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "courtyard" | entity_state=2 temporal=0 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "courtyard" | entity_state=2 temporal=0 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "photo studio" | entity_state=2 temporal=0 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "boiler room" | entity_state=9 temporal=0 coreference=1 distractors=2 context_length=13 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "cafeteria" | entity_state=9 temporal=0 coreference=1 distractors=2 context_length=13 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "boiler room" | entity_state=9 temporal=0 coreference=1 distractors=2 context_length=13 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "storage room" | entity_state=9 temporal=0 coreference=1 distractors=2 context_length=13 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "loading dock" | entity_state=16 temporal=0 coreference=1 distractors=2 context_length=20 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "observation deck" | entity_state=9 temporal=0 coreference=1 distractors=2 context_length=13 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "east wing" | entity_state=16 temporal=0 coreference=1 distractors=2 context_length=20 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "Cafeteria" | entity_state=16 temporal=0 coreference=1 distractors=2 context_length=20 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "south stairwell" | entity_state=16 temporal=0 coreference=3 distractors=2 context_length=20 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "Archive Room" | entity_state=23 temporal=0 coreference=3 distractors=2 context_length=27 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "mail room" | entity_state=16 temporal=0 coreference=1 distractors=2 context_length=20 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "west wing" | entity_state=23 temporal=0 coreference=1 distractors=2 context_length=27 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "atrium" | entity_state=23 temporal=0 coreference=1 distractors=2 context_length=27 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "photo studio" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "mail room" | entity_state=23 temporal=0 coreference=1 distractors=2 context_length=27 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "photo studio" | entity_state=30 temporal=0 coreference=3 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "storage room" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "print room" | entity_state=23 temporal=0 coreference=1 distractors=2 context_length=27 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "rooftop terrace" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "east wing" | entity_state=30 temporal=0 coreference=3 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "photo studio" | entity_state=30 temporal=0 coreference=3 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "cafeteria" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "south stairwell" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "print room" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → | |
| state | ✓PASSanswered "supply closet" | entity_state=30 temporal=0 coreference=1 distractors=2 context_length=34 witness_gap=0 | camp-full_map-ultra-002/state | transcript → |