Run camp-tool_bp-gemma-001
47% PASSWhat this run revealed, then every individual trial as evidence.
154
Trials
47%
72/154 scored
over trials
Pass rate
82
Failures
$0.0193
Run cost
Run camp-tool_bp-gemma-001: 47% pass rate on tool queries.
Of 154 scored trials (tool query kind), the model passed 47% and failed 53%.
82 failure(s) recorded; failures not yet verified -- use "Verify failures" on this page (or scripts/verify_past_failures.py).
Notable failures
- record 2c8613e3 expected: (unknown)actual: ```
- record f0919f45 expected: (unknown)actual: ```
- record 27282dbc expected: (unknown)actual: ```
Trials
Per-trial table
| model | verdict | kind | difficulty | cost ($) | tokens in/out | |
|---|---|---|---|---|---|---|
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000079 | 563/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 561/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 560/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000079 | 563/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 559/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000109 | 721/124 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000109 | 724/122 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000117 | 727/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000109 | 721/124 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000109 | 721/124 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000110 | 725/124 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 726/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 725/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000117 | 727/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000150 | 974/176 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000149 | 970/172 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000163 | 965/220 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 968/224 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000163 | 969/221 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000163 | 968/222 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000163 | 968/222 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000149 | 970/174 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000156 | 969/197 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000156 | 969/198 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000162 | 967/217 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 976/220 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 967/224 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 978/221 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 723/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 726/146 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000116 | 721/146 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000109 | 721/124 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000148 | 970/171 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000157 | 970/200 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 978/220 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 981/218 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000163 | 973/219 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000166 | 987/223 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000163 | 972/219 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000161 | 966/215 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000157 | 973/198 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000163 | 976/218 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000164 | 971/222 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000150 | 975/174 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000165 | 976/225 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000150 | 970/175 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 561/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 559/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000079 | 563/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 557/74 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000151 | 985/176 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000166 | 981/226 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000165 | 986/221 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000158 | 983/199 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000167 | 987/227 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000151 | 985/175 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000165 | 988/220 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000159 | 958/210 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000145 | 954/164 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000152 | 954/189 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000146 | 964/164 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000158 | 954/210 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000158 | 952/210 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000158 | 960/208 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000158 | 915/223 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000156 | 895/220 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000155 | 901/217 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000156 | 898/220 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000154 | 904/211 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000160 | 915/227 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000149 | 899/197 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000131 | 852/152 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000132 | 859/155 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000134 | 868/159 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000131 | 851/154 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000131 | 850/153 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000133 | 856/157 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000133 | 857/159 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000133 | 857/158 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000133 | 863/155 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000132 | 854/155 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000133 | 861/158 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000130 | 846/150 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000133 | 856/157 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000132 | 864/152 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000160 | 933/221 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000159 | 927/222 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000145 | 933/171 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000161 | 933/225 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000147 | 941/175 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000161 | 931/227 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000159 | 933/219 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000129 | 845/149 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000127 | 832/147 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000128 | 837/147 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000128 | 837/149 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000130 | 850/151 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000129 | 840/150 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000112 | 831/97 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000166 | 975/227 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000172 | 972/248 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000178 | 978/268 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000173 | 973/251 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000160 | 963/214 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000150 | 973/175 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=1)unverified | tool_plan | - | 0.000177 | 972/267 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000091 | 549/120 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000090 | 544/120 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000084 | 548/97 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000091 | 548/120 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000078 | 554/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000077 | 544/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000091 | 549/120 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000089 | 568/106 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000080 | 567/79 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000082 | 573/81 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000079 | 566/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000081 | 571/79 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000081 | 568/81 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000081 | 568/81 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000090 | 679/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000090 | 679/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000090 | 680/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000090 | 679/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000091 | 683/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000091 | 683/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000090 | 679/74 | full transcript → |
| google/gemma-3-27b-it | ✓PASSanswered "```" | tool_plan | - | 0.000109 | 683/136 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000102 | 683/114 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000103 | 685/114 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000103 | 685/114 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000102 | 683/114 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000103 | 686/114 | full transcript → |
| google/gemma-3-27b-it | ?FAILanswered "```" (world seeded for: poison_traps=0)unverified | tool_plan | - | 0.000103 | 686/114 | full transcript → |