II · Tool planning

Notices when a task is impossible

Nemotron 3 Super 120B

Finding:HOLDS· grade C(research conclusion)

Declines 10 of 12 impossible tasks; writes a plan for the other 2

Pass ratesingle rate, no difficulty dial

83%

10/12 scored trials passed · 95% interval 55%–95%

What the few failures have in common

The model writes a plan for an impossible goal instead of declining.

Example failure

Trial result: FAIL (one model answer)

FAILfabricated_plan_on_unsolvable:5

[{"tool":"advance_0","args":{"service":"svc_0"}},{"tool":"set_flag_0","args":{"flag":"flag_0"}},{"tool":"build_art_0","args":{"artifact":"art_0"}},{"tool":"publish_art_0","args":{"artifact":"art_0"}},{"tool":"advance_2","args":{"service":"svc_0"}}]

Open this trial →Open the run it came from →

Evidencewhat this finding rests on

Ground truthDeterministic simulation: the right answer is computed, not judged
StatisticsWilson 95% interval 55%–95%Grade C: thin evidence
Sample size12 scored trials in total: small sample (under 15)
Fresh-seed confirmationNot run yet
ReplicationNot run yet

Raw evidenceruns and trials

Runs behind this row
- camp-irrelevance-super-001 12 trials
Every tool-planning trial for this model → broader than this row
The example failure, in full →

Back to the capability map →