Run camp-tool_bp-gemma4qat-001

Run camp-tool_bp-gemma4qat-001: 70% pass rate on tool queries.

Of 245 scored trials (tool query kind), the model passed 70% and failed 30%.

73 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).

Sharp boundary detected on: catalog_size (high-end pass rate 50%), realized_plan_depth (high-end pass rate 20%).

Diffuse weakness (below 90% but no outright flip) on: dishonest_decoys_on (pass rate 56% at the high end), implicit_preconditions_on (pass rate 55% at the high end), irreversibility (pass rate 53% at the high end), parallel_goals (pass rate 51% at the high end), poison_traps (pass rate 53% at the high end), zero_slack_on (pass rate 53% at the high end).

Notable failures

• record 48d7673d — asked "You may use at most 9 tool calls."
  expected: (unknown)
  actual: ]

  answer did not match the verified ground truth

• record 108a2a07 — asked "You may use at most 10 tool calls."
  expected: (unknown)
  actual: ]

  answer did not match the verified ground truth

• record d5834920 — asked "You may use at most 10 tool calls."
  expected: (unknown)
  actual: ]

  answer did not match the verified ground truth