Most agentic RL approaches rely on costly external or synthesized executable environments, which bottleneck scale and private experimentation. EnvACE flips that constraint: during training the policy alternates between acting (producing tool calls) and playing the environment (rehearsing the response), and both roles are optimized end-to-end with task-success rewards. The result is an internalized agent world model that directly supports decision making and private rehearsal before committing to external execution.
Key Findings
- World rehearsal enables the policy to internalize action→response mappings so the agent needs fewer external interactions at train time, which reduces dependency on verified simulators and environment construction overhead; this improves sample efficiency in practice.
- Across multiple benchmarks (BFCL-v4, tau^2-Bench, VitaBench, FinMCP-Bench) EnvACE shows stronger and more transferable performance than environment-scaling baselines, and controlled ablations confirm consistent gains across model scales.
- The learned internal world model supports limited private rehearsal at test time: modest rehearsal budgets yield additional improvements without extra environment calls, useful for cost- or privacy-sensitive deployment scenarios.
- The method is end-to-end and model-agnostic: the same rehearsal mechanism applies across LLM agent scales and can be combined with standard RL objectives.
Who it's for + Tradeoffs
Great fit if you train LLM-based agents for long-horizon tool use but have limited access to high‑fidelity simulators or need private validation before execution. EnvACE reduces reliance on external environment pipelines and can speed iteration for research and applied agent deployment. Look elsewhere if your application mandates provable real-world grounding at train time (e.g., safety-critical systems requiring real environment verification) or when simulator fidelity is the primary bottleneck—EnvACE internalizes dynamics but does not replace the need to validate real-world execution in high-stakes settings.
Where it fits
EnvACE sits between simulator-heavy environment-scaling approaches and pure outcome-based RL: it trades some fidelity of real environment interaction for greatly reduced environmental engineering and the ability to rehearse privately. The authors release code for reproduction and to facilitate integration into LLM-agent training pipelines.