Trials · temporal (high)
Every trial behind one square of the capability map: Gemma 4 31B QAT on temporal (high).
266
0 errors
Trials in this cell
51%
135/266 scored
Pass rate
131
0 noise excluded
Real failures
$0.0126
Cell cost
Trials
| kind | verdict | realized dims | seeded traps | run | |
|---|---|---|---|---|---|
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=0 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.60 coreference=0 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.60 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=0 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✕FAILanswered "Since neither event described in your question actual..."verified-stable | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.60 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-full_map-gemma4qat-001/order | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=2 temporal=0.24 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "loading dock" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "library" | entity_state=2 temporal=0.24 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "archive room" | entity_state=2 temporal=0.48 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "security office" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "front office" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "workshop" | entity_state=2 temporal=0.48 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "reception area"unverified | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "basement"unverified | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "cafeteria"unverified | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "observation deck" | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "loading dock" | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=2 temporal=0.71 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "security office"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "utility closet" | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ?FAILanswered "south stairwell"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-full_map-gemma4qat-001/belief | transcript → | |
| belief | ✓PASSanswered "front office" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "break room" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "basement" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "print room" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "basement" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "storage room"unverified | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "conference room" | entity_state=2 temporal=0.24 coreference=0 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "greenhouse"unverified | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "garage" | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "library" | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "photo studio"unverified | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "boiler room"unverified | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "break room"unverified | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "archive room" | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "archive room"unverified | entity_state=2 temporal=0.71 coreference=0 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "photo studio"unverified | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ?FAILanswered "archive room"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "gym" | entity_state=2 temporal=0.71 coreference=0 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✕FAILanswered "north stairwell"verified-stable | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-full_map-gemma4qat-001/adversarial_belief | transcript → |
| belief | ✓PASSanswered "cafeteria" | entity_state=3 temporal=0.58 coreference=3 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "supply closet" | entity_state=3 temporal=0.51 coreference=3 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "lobby" | entity_state=3 temporal=0.58 coreference=3 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=3 temporal=0.51 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "reception area"unverified | entity_state=3 temporal=0.58 coreference=3 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "photo studio" | entity_state=3 temporal=0.55 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "west wing"unverified | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "front office"unverified | entity_state=6 temporal=0.51 coreference=3 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "boiler room"unverified | entity_state=6 temporal=0.47 coreference=2 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "courtyard" | entity_state=6 temporal=0.51 coreference=2 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=6 temporal=0.47 coreference=2 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "security office" | entity_state=6 temporal=0.51 coreference=3 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=6 temporal=0.51 coreference=3 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "observation deck"unverified | entity_state=6 temporal=0.49 coreference=3 distractors=9 context_length=30 witness_gap=5 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "library" | entity_state=5 temporal=0.51 coreference=3 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "boiler room" | entity_state=5 temporal=0.48 coreference=2 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "workshop" | entity_state=5 temporal=0.50 coreference=3 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "cafeteria"unverified | entity_state=5 temporal=0.51 coreference=3 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "greenhouse"unverified | entity_state=5 temporal=0.48 coreference=3 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "server room"unverified | entity_state=5 temporal=0.51 coreference=3 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "workshop" | entity_state=5 temporal=0.51 coreference=3 distractors=6 context_length=21 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "east wing"unverified | entity_state=5 temporal=0.50 coreference=3 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "print room" | entity_state=5 temporal=0.50 coreference=2 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "conference room"unverified | entity_state=5 temporal=0.50 coreference=3 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "security office"unverified | entity_state=5 temporal=0.50 coreference=6 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=5 temporal=0.50 coreference=6 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "gym"unverified | entity_state=5 temporal=0.50 coreference=2 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=5 temporal=0.50 coreference=3 distractors=8 context_length=26 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "loading dock"unverified | entity_state=5 temporal=0.51 coreference=2 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "storage room" | entity_state=5 temporal=0.51 coreference=2 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "utility closet"unverified | entity_state=5 temporal=0.49 coreference=2 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "kitchen"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "loading dock"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "print room"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=5 temporal=0.51 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "boiler room"unverified | entity_state=5 temporal=0.51 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=5 temporal=0.51 coreference=6 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "courtyard"unverified | entity_state=5 temporal=0.49 coreference=3 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "greenhouse"unverified | entity_state=5 temporal=0.51 coreference=2 distractors=7 context_length=23 witness_gap=4 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "storage room" | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "north stairwell" | entity_state=3 temporal=0.55 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "break room" | entity_state=3 temporal=0.51 coreference=3 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "storage room"unverified | entity_state=3 temporal=0.51 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "gym" | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "server room"unverified | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "supply closet"unverified | entity_state=3 temporal=0.58 coreference=3 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ✓PASSanswered "reception area" | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "reception area"unverified | entity_state=3 temporal=0.51 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=3 temporal=0.51 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| belief | ?FAILanswered "server room"unverified | entity_state=3 temporal=0.58 coreference=1 distractors=4 context_length=11 witness_gap=2 | camp-full_map-gemma4qat-001/breaking_point | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.26 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.23 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.27 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.29 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "rooftop terrace"unverified | entity_state=5 temporal=0.15 coreference=2 distractors=8 context_length=26 witness_gap=1 | camp-breaking_point-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.30 coreference=0 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.29 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "storage room" | entity_state=5 temporal=0.15 coreference=3 distractors=8 context_length=26 witness_gap=1 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "break room" | entity_state=9 temporal=0.27 coreference=4 distractors=13 context_length=46 witness_gap=3 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "cafeteria" | entity_state=9 temporal=0.27 coreference=4 distractors=13 context_length=46 witness_gap=3 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=2 temporal=0.24 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "loading dock" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "break room" | entity_state=9 temporal=0.27 coreference=6 distractors=13 context_length=46 witness_gap=3 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "front office" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ✓PASSanswered "basement" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "library" | entity_state=2 temporal=0.24 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "break room" | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ?FAILanswered "unknown"unverified | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "archive room" | entity_state=2 temporal=0.48 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "wine cellar"unverified | entity_state=12 temporal=0.36 coreference=9 distractors=19 context_length=66 witness_gap=5 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "storage room"unverified | entity_state=2 temporal=0.24 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ✓PASSanswered "conference room" | entity_state=2 temporal=0.24 coreference=0 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ✓PASSanswered "print room" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ✓PASSanswered "front office" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.27 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "basement" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ✓PASSanswered "workshop" | entity_state=2 temporal=0.48 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "security office" | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "reception area"unverified | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "reception area"unverified | entity_state=12 temporal=0.37 coreference=11 distractors=19 context_length=66 witness_gap=5 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "greenhouse"unverified | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ?FAILanswered "cafeteria"unverified | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "break room"unverified | entity_state=2 temporal=0.48 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.53 coreference=0 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "garage" | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ?FAILanswered "basement"unverified | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "unknown"unverified | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "observation deck" | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "wine cellar"unverified | entity_state=12 temporal=0.37 coreference=11 distractors=19 context_length=66 witness_gap=5 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "boiler room"unverified | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.60 coreference=0 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "library" | entity_state=2 temporal=0.67 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ✓PASSanswered "loading dock" | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=3 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.60 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "archive room" | entity_state=2 temporal=0.67 coreference=3 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ?FAILanswered "security office"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "utility closet" | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=2 temporal=0.71 coreference=3 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "garage"unverified | entity_state=11 temporal=0.32 coreference=15 distractors=16 context_length=57 witness_gap=4 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "archive room"unverified | entity_state=2 temporal=0.71 coreference=0 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.42 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "gym" | entity_state=2 temporal=0.71 coreference=0 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "north stairwell"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "south stairwell"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | camp-belief-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "archive room"unverified | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.73 coreference=0 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "photo studio"unverified | entity_state=11 temporal=0.32 coreference=7 distractors=16 context_length=57 witness_gap=4 | camp-breaking_point-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=2 temporal=0.60 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "rooftop terrace" | entity_state=2 temporal=0.71 coreference=1 distractors=2 context_length=7 witness_gap=1 | final_true_statelast_mentioned_locationlast_witnessed_by_anyone | camp-adversarial_belief-gemma4qat-001 | transcript → |
| belief | ?FAILanswered "garage"unverified | entity_state=11 temporal=0.32 coreference=7 distractors=16 context_length=57 witness_gap=4 | camp-breaking_point-gemma4qat-001 | transcript → | |
| order | ?FAILanswered "Since neither event described in your question actual..."unverified | entity_state=2 temporal=0.73 coreference=1 distractors=2 context_length=6 witness_gap=0 | camp-order-gemma4qat-001 | transcript → | |
| belief | ✓PASSanswered "photo studio" | entity_state=10 temporal=0.29 coreference=6 distractors=15 context_length=53 witness_gap=3 | camp-breaking_point-gemma4qat-001 | transcript → | |
| order | ?FAILanswered "Marcus moving the tablet to the photo studio (8:04) h..."unverified | entity_state=8 temporal=0.41 coreference=1 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.41 coreference=3 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "boiler room"unverified | entity_state=10 temporal=0.29 coreference=10 distractors=15 context_length=53 witness_gap=3 | camp-breaking_point-gemma4qat-001 | transcript → | |
| belief | ?FAILanswered "wine cellar"unverified | entity_state=10 temporal=0.30 coreference=7 distractors=15 context_length=53 witness_gap=3 | camp-breaking_point-gemma4qat-001 | transcript → | |
| order | ✓PASSanswered "first" | entity_state=8 temporal=0.52 coreference=3 distractors=2 context_length=12 witness_gap=0 | camp-premise-order-gemma4qat-001 | transcript → |