Run smoke-gemma-tool

FAILED

What this run revealed, then every individual trial as evidence.

Trials

0/1 scored

Pass rate

100%

Failures

$0.0001

Run cost

Run smoke-gemma-tool: 0% pass rate on tool queries.

Of 1 scored trials (tool query kind), the model passed 0% and failed 100%.

1 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

record 956e92d6 — asked "You may use at most 14 tool calls."
expected: (unknown)
actual: ```
answer did not match the verified ground truth

Trials1 trial

Per-trial table (1 trials -- prompt, reasoning, verdict, answer key)

model	verdict	kind	difficulty	cost ($)	tokens in/out
google/gemma-3-27b-it	?FAILanswered "```"unverified	tool_plan	-	0.000121	771/146	full transcript →