A coding model’s benchmark score can depend as much on how it uses tools as on the underlying model. This adapter targets that interaction: it trains Qwen3.8-27B for pi, with reported gains in task completion alongside fewer tool calls. The results are promising, but they are harness-specific and should be read with the checkpoint-selection process in mind.
Key Capabilities
- On Terminal-Bench 2.1, the reported pass rate rises from 39.5% to 55.8%, while tool calls per task fall by 62%. A 68-task subset not used for checkpoint selection shows a similar pass-rate gain, offering a useful—but not fully independent—check.
- On the 51 hardest SWE-bench Pro tasks, the reported rate increases from 29.4% to 62.7%, with 55% fewer tool calls per task. These results are for a hard subset, not the full benchmark.
- The adapter is trained with AsyncGRPO and LoRA using pi agent trajectories and a reward that combines task correctness with a tool-efficiency bonus. It must be paired with Qwen3.8-27B; it is not a general-purpose adapter for other base models.
Who It’s For—and the Trade-Off
A good fit for developers evaluating or running Qwen3.8-27B as a pi-based coding agent, especially on terminal and repository tasks. Treat the published figures as creator-reported results: checkpoint step 160 was selected using SWE-bench Pro scores and an 18-task Terminal-Bench subset, so the SWE-bench result is not an untouched test. Look elsewhere if you need an independently verified score, results across different agent harnesses, or a drop-in adapter for another base model.