Most progress on LLM coding agents treats either the model or the harness as fixed. That misses a simple insight: the harness determines which behaviors a model can reliably express, and the model’s failures change after each policy update. Self-HPO flips this by letting the policy propose harness edits and then training the policy under the selected harness in a recursive loop, keeping optimization focused on the policy’s current blind spots.
Key Findings
- Alternating harness search and policy learning produces complementary gains: harnesses expose hard behaviors, and RL internalizes them into the model, so the joint improvement exceeds optimizing either component alone. This means you get better net progress per optimization round than standard harness-only or policy-only runs.
- Empirical lifts on coding benchmarks: applied to Qwen3.5-35B-A3B and the mini-SWE-agent harness, Self-HPO raised average resolve rates (e.g., from 57.8% to 64.8% overall, and higher on SWE-Bench Verified), with improvements transferring to out-of-distribution multilingual and pro benchmark variants. So the method improves both capability and robustness.
- Design keeps harness search targeted: each H-step analyzes failed trajectories from the current policy to propose minimal, diverse harness modifications; the P-step then trains under the chosen harness and surfaces hard tasks for the next H-step. Practically, this focuses search budget on evolving failure modes rather than a static set of cases.
- External proposers help but are optional: using a stronger external model to propose harness candidates (e.g., Opus 4.8) can further boost performance, indicating the framework is compatible with either self-proposal or guided proposal strategies.
Who it's for + tradeoffs
Great fit if you develop or evaluate coding agents and want a pragmatic way to push a model’s capability frontier without manual harness engineering: Self-HPO automates targeted harness revisions and pairs them with RL updates to iteratively close gaps. Look elsewhere if you lack a reliable executable verifier or face strict compute limits: the method relies on on-policy data collection, verifiable rewards, and repeated harness search/optimization cycles, which add compute and evaluation overhead compared to one-shot tuning.
Where it fits
Self-HPO sits between harness-only prompt/workflow search and pure policy training: use it when you can instrument agent execution (a harness) and run verifiable tests so that hard cases can be identified, proposed, and then used to steer further policy learning.