Trials · code_repair

Every trial behind one square of the capability map: Gemma 4 31B QAT on code_repair.

Gemma 4 31B QAT code_repair unverified

← back to this model's findings

7 errors

Trials in this cell

33%

22/67 scored

Pass rate

0 noise excluded

Real failures

$0.0367

Cell cost

Trials22/6733%

kind	verdict	run
code_repair	?FAILanswered "```"unverified	smoke-v-gemma4qat	transcript →
code_repair	✓PASSanswered "```"	smoke-v-gemma4qat	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-001	transcript →
code_repair	✕ERRORstop	camp-code-repair-kinds-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-001	transcript →
code_repair	✕ERRORtimeout	camp-code-repair-kinds-gemma4qat-001	transcript →
code_repair	✕ERRORstop	camp-code-repair-kinds-gemma4qat-002	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-002	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-002	transcript →
code_repair	✕ERRORtimeout	camp-code-repair-kinds-gemma4qat-002	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-002	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✕ERRORstop	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✕ERRORtimeout	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	✕ERRORstop	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-kinds-gemma4qat-postfix-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-postfix-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-confirm-gemma4qat-postfix-001	transcript →
code_repair	✓PASSanswered "```"	camp-code-repair-kinds-gemma4qat-postfix-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-postfix-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-postfix-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-code-repair-confirm-gemma4qat-postfix-001	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/serial	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	✓PASSanswered "```"	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →
code_repair	?FAILanswered "```"unverified	camp-determinism-probe-gemma4qat-001/parallel	transcript →