Run camp-full_map-ultra-002/order
PASSEDWhat this run revealed, then every individual trial as evidence.
80
Trials
100%
80/80 scored
over trials
Pass rate
0
Failures
$0.0856
Run cost
Run camp-full_map-ultra-002/order: passed essentially everything (100%) -- this test did not challenge the model.
Of 80 scored trials (order query kind), the model passed 100%. That's saturation: the difficulty range probed here isn't finding this model's limits. Consider a harder preset (breaking_point or adversarial_belief) or a wider dimension range.
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000632 | 160/251 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000868 | 156/359 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000992 | 153/416 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001039 | 155/437 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001506 | 151/650 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001824 | 221/779 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000625 | 229/232 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001208 | 225/498 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001125 | 225/460 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000579 | 308/193 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001610 | 227/680 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001351 | 321/541 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000540 | 309/175 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000730 | 430/234 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000834 | 413/285 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001624 | 309/668 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001619 | 329/661 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001039 | 406/380 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000584 | 411/172 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001907 | 400/776 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000712 | 487/213 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001290 | 481/477 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001349 | 480/504 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001050 | 468/371 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001254 | 471/463 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000424 | 152/158 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000634 | 145/255 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000962 | 151/403 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001088 | 156/459 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001304 | 157/557 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001307 | 154/559 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000619 | 151/247 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000857 | 152/355 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000530 | 150/207 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000712 | 156/288 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001273 | 161/542 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001555 | 154/672 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001167 | 151/496 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001106 | 149/469 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001368 | 153/587 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001412 | 148/608 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000648 | 153/260 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001863 | 161/810 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000738 | 156/300 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001500 | 154/647 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000937 | 157/390 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001371 | 150/589 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000454 | 164/169 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001508 | 156/650 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.002522 | 155/1111 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000395 | 157/144 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000831 | 149/344 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000931 | 160/387 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000787 | 153/323 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001322 | 153/566 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000694 | 288/250 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000611 | 417/183 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001362 | 290/553 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000716 | 279/262 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.002345 | 277/1003 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001999 | 289/843 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000633 | 391/199 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000644 | 399/202 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000748 | 405/248 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000779 | 409/261 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001232 | 511/444 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000725 | 518/212 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000661 | 548/176 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000609 | 554/151 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000823 | 529/254 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000833 | 641/233 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001055 | 666/328 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000833 | 668/227 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000744 | 656/189 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001628 | 620/599 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000371 | 156/133 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.000707 | 152/287 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001005 | 153/422 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.001808 | 161/785 | full transcript → |
| nvidia/Nemotron-3-Ultra-550b-a55b | ✓PASSanswered "first" | order | - | 0.002056 | 151/900 | full transcript → |