Static, hand-crafted environments often become limiting as LLM agents improve: they conceal recurring failure modes, require costly domain-specific rebuilds, and yield weak signals for continued training. EnvHarness reframes the problem by adding a programmable harness layer that sits around an environment and reshapes its observed behavior while preserving the original verifier. EnvRigger automates this loop by treating the target policy as a black box, diagnosing failures from execution traces, synthesizing harness components, and validating them through reruns so harness edits are grounded in realized outcomes.
Key Findings
-
Programmable harness layer: EnvHarness wraps environments via standard interfaces so reshaped instances keep the original verifier and can be applied across diverse domains — this reduces the need for bespoke environment engineering.
-
Automated, trajectory-driven synthesis: EnvRigger converts failure trajectories into reusable harness components and validates them with fresh rollouts, eliminating many manual, domain-specific pipelines and unreliable external verifiers.
-
Measurable empirical gains: Across five benchmarks in four domains, EnvHarness achieves up to +9.0 percentage points on held-out instances and 9.8% fewer execution steps, indicating both higher success and greater execution efficiency.
-
Better RL signal and co-evolution: The harnessed environments provide a stronger optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of policy and environment without modifying simulator internals.
Who it's for and tradeoffs
Great fit if you run LLM-based agents in hand-built or legacy simulators and want to (a) address recurring failure modes without rewriting environments, (b) automate harness updates from real trajectories, and (c) accelerate policy learning via targeted environment edits. Look elsewhere if your evaluation requires an untouched ground-truth simulator (no runtime intervention), if environment changes must be reflected in the simulator codebase itself, or if adding an external harness would violate reproducibility constraints in your workflow. EnvHarness simplifies iteration and domain transfer at the cost of adding a runtime adaptation layer that needs its own validation and maintenance.