AIAny
Icon for item

OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

Evolves persistent, stateful environments to red-team tool-using AI agents — provides 10K+ validated scenarios across 50 domains and a feedback-driven attack policy (EMHA) to surface long‑horizon safety failures.

Introduction

Early environment changes can steer agent behavior far into the future; evaluating agents only on short, static tasks misses these cumulative effects. This work operationalizes that insight by treating the environment state as the primary red‑teaming unit and by evolving target‑visible state (while keeping objectives and hidden safety contracts fixed) to systematically increase attack difficulty without changing semantics.

Key Findings
  • Scalable dataset: generates and validates over 10,000 stateful scenarios spanning 50 domains, synthesized from a corpus of 500K+ Tools, MCPs, and Skills. Tasks are long‑horizon (median 97 tool calls) and target agent–model pairs through lightweight adapters to form 75 agent–model configurations.
  • EMHA attack policy: introduces Evolutionary Markov Hypergraph Attack (EMHA), a black‑box, feedback‑driven policy that coordinates authorized state transitions along hypergraph paths to evolve environments without model parameter updates.
  • Empirical impact: across all configurations, EMHA attains a pooled strict Attack Success Rate (ASR) of 85.0%. Environment evolution yields small gains on simple tasks (≈1.8–2.7%) but large gains on the most complex tasks (≈17.2–17.6%), showing long‑horizon workflows amplify the effects of state evolution.
  • Implementation matters: after controlling for model and benign task completion, target‑agent identity explains an additional 7.6% of ASR variation, indicating runtime implementations significantly influence safety outcomes.
Who it's for and tradeoffs

Great fit if you evaluate or develop tool‑using agents and need a unified, high‑complexity benchmark to surface cumulative, stateful failures. The arena is useful for red‑teamers, safety researchers, and teams comparing runtime implementations under identical task semantics. Look elsewhere if you need real‑world deployment monitoring (OpenART focuses on controlled, executable scenarios rather than production telemetry), or if your target agents cannot be instrumented to support the lightweight adapters and tool/skill interfaces the benchmark assumes.

Where it fits

OpenART complements short, static benchmarks by focusing on persistent state evolution and long interaction horizons. It is positioned as a scalable research platform and benchmark rather than a turnkey production safety system.

How EMHA and environment evolution work

EMHA models environment evolution as hypergraph traversal: evaluator feedback authorizes or rejects candidate state transitions, and EMHA iteratively refines future transitions using past success signals. The pipeline keeps benign objectives and hidden safety contracts fixed while only mutating the target‑visible state, enabling controlled difficulty escalation without altering task semantics.

Information

  • Websitearxiv.org
  • OrganizationsAffiliation: Fudan University, Affiliation: Shanghai Artificial Intelligence Laboratory, Affiliation: XSafeAI
  • AuthorsYunhao Chen, Xin Wang, Yixu Wang, Yi Liu, Jie Li, Yan Teng, Xingjun Ma, Xia Hu, Yu-Gang Jiang
  • Published date2026/08/01

Categories

More Items

Evaluates whether LLM-driven storyteller agents preserve long-horizon logical consistency under adversarial player interventions. Introduces NCP-Bench (100 movie-derived narrative environments) with structured trajectory/commitments and automatic violation checks; finds strong LLMs often contradict themselves across multi-turn interactions.

Uses a stronger 'builder' model at inference time to construct executable harnesses that boost weaker target models without parameter updates, mainly by turning unstable reasoning into deterministic code, routing, and strict answer-format enforcement.

Enables a coding agent to self-develop by evolving its harness, prompts, tools, and core code via reviewed commits — supporting recursive free evolution and experience-driven evolution. Demonstrated a 161-day live lineage and state-of-the-art scores on multiple coding benchmarks while foregrounding operational safety.