Run camp-full_map-gemma-001/order
99% PASSWhat this run revealed, then every individual trial as evidence.
80
Trials
99%
79/80 scored
over trials
Pass rate
1
1 stable / 0 noise
Failures
$0.0047
Run cost
Run camp-full_map-gemma-001/order: passed essentially everything (99%) -- this test did not challenge the model.
Of 80 scored trials (order query kind), the model passed 99%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
1 failures recorded; all 1 survived verification.
Notable failures
- record d73cbb84 expected: firstactual: **second**
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, Bruno moving the box happened first." | order | - | 0.000033 | 149/59 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:04, so Oscar's action happened first." | order | - | 0.000034 | 146/66 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000035 | 142/69 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000048 | 213/90 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, Maya's action happened first." | order | - | 0.000032 | 152/56 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000056 | 219/113 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:04, so Sana moving the tablet to the bre..." | order | - | 0.000035 | 144/69 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000053 | 216/103 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:08, so Noor's action happened first." | order | - | 0.000043 | 218/69 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000047 | 299/56 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000058 | 302/94 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000064 | 313/108 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000052 | 298/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000051 | 311/66 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000066 | 391/91 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000069 | 394/98 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000108 | 217/289 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**first**" | order | - | 0.000072 | 469/83 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000077 | 472/99 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, Kira moving the box happened first." | order | - | 0.000064 | 461/60 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000061 | 401/69 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000074 | 467/90 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000069 | 400/97 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000081 | 461/116 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000075 | 404/115 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**first**" | order | - | 0.000034 | 143/64 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000039 | 146/81 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000042 | 145/90 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000041 | 140/91 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:05, so moving the box happened first." | order | - | 0.000031 | 146/56 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Dana's action happened at 8:00, which is before Wyatt's a..." | order | - | 0.000038 | 146/79 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000069 | 149/180 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000047 | 143/108 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000067 | 146/174 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000079 | 151/214 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000065 | 143/170 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000069 | 149/179 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000078 | 147/210 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000035 | 138/72 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "first" | order | - | 0.000015 | 141/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000068 | 142/179 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000068 | 150/176 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**first**" | order | - | 0.000060 | 148/151 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**first**" | order | - | 0.000064 | 149/164 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000082 | 145/226 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Since 8:00 is earlier than 8:05, Sana moving the folder h..." | order | - | 0.000036 | 152/68 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000064 | 142/167 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000066 | 150/170 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000069 | 148/182 | full transcript → |
| google/gemma-3-27b-it | ✕FAILanswered "**second**"verified-stable | order | - | 0.000062 | 143/159 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:04, so moving the folder happened first." | order | - | 0.000034 | 147/63 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000037 | 142/76 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000050 | 263/79 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000043 | 142/97 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000041 | 280/45 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000047 | 272/65 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000054 | 280/86 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "The answer is **first**." | order | - | 0.000039 | 148/79 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000075 | 274/160 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**first**" | order | - | 0.000046 | 143/105 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000063 | 386/83 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000071 | 401/102 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000067 | 394/92 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: first" | order | - | 0.000065 | 381/89 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000082 | 506/106 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000081 | 513/98 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000091 | 543/122 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000078 | 524/85 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:32, so Willa moving the box happened first." | order | - | 0.000087 | 653/71 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000088 | 627/84 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000077 | 526/83 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: **first**" | order | - | 0.000093 | 651/93 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000101 | 660/115 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000068 | 394/94 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Answer: first" | order | - | 0.000088 | 612/90 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "first" | order | - | 0.000016 | 147/3 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "8:00 is before 8:05, so Finn moving the folder to the uti..." | order | - | 0.000033 | 148/62 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000047 | 146/109 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "**Answer: first**" | order | - | 0.000036 | 152/71 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "Therefore, the answer is **first**." | order | - | 0.000039 | 144/81 | full transcript → |