Data to …… trials
II · Tool planning

Picks the right tool from a large catalog

Nemotron 3 Super 120B

Finding:HOLDS· grade A(research conclusion)

Holds through 66-tool catalogs: 112/112 pass across every tested level

Held through 66-tool catalogs

Pass rate by level8 levels tested
Pass rate by level for Picks the right tool from a large catalog0%30%50%75%100%8-tool catalogs8-tool catalogs: 14/14 pass (100%)14/1417-tool catalogs17-tool catalogs: 14/14 pass (100%)14/1425-tool catalogs25-tool catalogs: 14/14 pass (100%)14/1434-tool catalogs34-tool catalogs: 14/14 pass (100%)14/1443-tool catalogs43-tool catalogs: 14/14 pass (100%)14/1452-tool catalogs52-tool catalogs: 14/14 pass (100%)14/1460-tool catalogs60-tool catalogs: 14/14 pass (100%)14/1466-tool catalogs66-tool catalogs: 14/14 pass (100%)14/14

Dashed lines: a level at or below 30% pass counts toward BREAKS, at or below 75% toward DEGRADES. Labels above points: passed/scored.

LevelPassedPass rateNote
8-tool catalogs14/14100%
17-tool catalogs14/14100%
25-tool catalogs14/14100%
34-tool catalogs14/14100%
43-tool catalogs14/14100%
52-tool catalogs14/14100%
60-tool catalogs14/14100%
66-tool catalogs14/14100%
What the few failures have in common

No failures recorded in the trials behind this row.

Evidencewhat this finding rests on
  • Ground truthDeterministic simulation: the right answer is computed, not judged
  • StatisticsWilson 95% interval 97%–100%Grade A: strong evidence
  • Sample size112 scored trials in total
  • Fresh-seed confirmationNot run yet
  • ReplicationNot run yet
Raw evidenceruns and trials

Back to the capability map →