Data to …… trials
Reliability

Gets the same task right every time (pass^k)

Nemotron 3 Nano 30B

Finding:NOT TESTED(no evidence yet)

Not tested yet

Rate shown is pass^1 over repeats of the same tasks (X.consistency campaign); pass^k needs per-task grouping and is in results/consistency/<run_id>.json. Earlier hand measurement: Nano 63% single-try vs 17% on all 5 tries.

Not tested yet: there is no evidence about this capability for this model in the store.

That gap is itself a result: it marks what we do not know.

Back to the capability map →