Most agent skill optimization stacks bundle many heuristics and slow meta‑updates. SkillOpt‑Lite challenges that orthodoxy: by treating execution traces as flat files and keeping the loop to trajectory exploration, consensus mining, and independent validation, the paper shows faster convergence and higher final scores — and in some cases a harness‑optimized smaller model beats a larger baseline.
Key Findings
- Minimal viable pipeline: removing reflection pooling, slow update damping, and rejection buffers yields faster refinement and higher validation ceilings. So what? Simpler tooling reduces early exploration plateaus and speeds discovery of robust edits.
- File-centric trajectory edits: isolating each rollout as a file and granting primitive file-system utilities leads to effective, semantic patches that can outperform multi-epoch heavy pipelines on logic tasks. So what? It makes debugging-style, human-interpretable edits the core optimization substrate instead of opaque param updates.
- Harness co-design (HarnessOpt): optimizing the execution harness alongside skill text flips capability rankings — e.g., on SpreadsheetBench HarnessOpt enables GPT‑5.4‑nano to reach 0.7758 accuracy vs. GPT‑5.5 at 0.7620 under a standard pipeline. So what? Environment and scaffolding matter as much as the skill text; co-optimization can let smaller models outcompete larger ones in suboptimal environments.
- Quantified wins: reported improvements include LiveMath +8.8 points on GPT‑5.5 and +25.4 points on GPT‑5.4‑nano versus full SkillOpt baselines, and joint harness+skill optimization yields peak scores (e.g., 0.8505 for GPT‑5.4, 0.8577 for GPT‑5.5).
Who this is for and tradeoffs
Great fit if you build or research LLM-based agents, need reproducible, low-overhead skill tuning, or want IDE-integrated loops (one-line VSCode command). Look elsewhere if you require differentiable weight updates, large-scale distributed training of model weights, or workflows that demand strict formal guarantees beyond held-out validation gating. SkillOpt‑Lite favors pragmatic, text-space edits and harness co-design over weight-space optimization.
Where it fits
Positioned between heavy text-space optimizers and ad hoc self-revision systems: it keeps the reproducibility discipline of SkillOpt but strips to the empirical essentials. Use it when you want fast iteration on agent behavior, harness edits, and interpretable patch artifacts that transfer across scales and execution environments.
Methodological note
The paper frames skill training as a zeroth‑order optimization problem where agent rollouts provide semantic perturbations rather than blind numerical probes. The three empirical principles (file-system trajectory exploration, consensus attribute mining, independent validation gating) are both theoretically motivated and validated across multiple benchmarks and model scales.