Data to …… trials
II · Tool planning

Notices when a task is impossible

Nemotron 3 Nano 30B

Finding:DEGRADES· grade C(research conclusion)

Declines 6 of 12 impossible tasks; writes a plan for the other 6

Pass ratesingle rate, no difficulty dial
50%

6/12 scored trials passed · 95% interval 25%–75%

breaks ≤ 30%holds if interval reaches 95%0%50%100%
Why it slips

The model writes a plan for an impossible goal instead of declining.

in 100% of failures (6 failures)

Example failure
Trial result: FAIL (one model answer)
FAILfabricated_plan_on_unsolvable:5
[ {"tool":"set_flag_0","args":{"flag":"flag_0"}}, {"tool":"build_art_0","args":{"artifact":"art_0"}}, {"tool":"advance_0","args":{"service":"svc_0"}}, {"tool":"advance_1","args":{"service":"svc_0"}}, {"tool":"advance_2","args":{"service":"svc_0"}} ]
Open this trial →Open the run it came from →
Evidencewhat this finding rests on
  • Ground truthDeterministic simulation: the right answer is computed, not judged
  • StatisticsWilson 95% interval 25%–75%Grade C: thin evidence
  • Sample size12 scored trials in total: small sample (under 15)
  • Fresh-seed confirmationNot run yet
  • ReplicationNot run yet
Raw evidenceruns and trials

Back to the capability map →