Data to …… trials

Trials · tool_plan

Every trial behind one square of the capability map: Nemotron 3 Nano 30B on tool_plan.

Nemotron 3 Nano 30B tool_plan unverified

649
23 errors
Trials in this cell
66%
413/626 scored
Pass rate
213
0 noise excluded
Real failures
$0.3569
Cell cost
Trials413/62666%
kindverdictrealized dimsseeded trapsrun
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan?FAILanswered "[{"tool":"provision","args":{"service":"svc_0"}}]"unverifiedcamp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}}]"camp-tool_planning-nvidia-nemotron-3-nano-30b-a3b-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "[{"tool":"build-_art_0","args":{"artifact":"art_0"}},..."unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "[{"tool":"build_art_0","args":{"artifact":"art_0"}},{..."unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=0camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✕ERRORlengthpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan?FAILanswered "]"unverifiedpoison_traps=1camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "[{"tool":"advance_0","args":{"service":"svc_0"}},{"to..."poison_traps=0camp-tool_bp-nano-001transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-propose-toolworld-001/round3transcript →
tool_plan✓PASSanswered "[{"tool":"build_art_0","args":{"artifact":"art_0"}},{..."poison_traps=1camp-propose-toolworld-001/round3transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-propose-toolworld-001/round3transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-propose-toolworld-001/round3transcript →
tool_plan✓PASSanswered "]"poison_traps=1camp-propose-toolworld-001/round3transcript →
tool_plan✓PASSanswered "[{"tool":"advance_o0","args":{"service":"svc_0"}},{"t..."poison_traps=0camp-propose-toolworld-001/round3transcript →

Showing the first 200 of 649 matching trials.