AIAny
Icon for item

LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks

Reformulates long-horizon agent execution as explicit task-state management: a manager defines bounded subtasks, fresh-context executors run them, and read-only auditors verify outcomes. Shows large performance gains on WeaveBench, Terminal-Bench and OSWorld.

Introduction

Most LLM agent harnesses let execution state accumulate in growing context windows, which lets incorrect self-assessments and stray context propagate across many steps. The core insight of this paper is that treating long-horizon execution as a task-state management problem — keeping a compact, verified state outside executor contexts and gating progress with independent audits — prevents error accumulation and turns intermediate progress into reliable completed work.

Key Findings
  • Explicit Manage–Execute–Audit (MEA) loop: a manager emits a bounded subtask contract, a fresh-context executor performs only that subtask, and a read-only auditor independently verifies environment changes before state updates. This enforces modular, verifiable progress.
  • Large empirical gains: the harness raises Qwen 3.7-Plus (with a Claude Code executor) from 51.8% to 80.7% on WeaveBench, improves Terminal-Bench success from 69.7% to 77.2% under matched settings, and increases OSWorld completion from 2.8% to 8.3%; Claude Opus sees similar relative gains on an OSWorld subset.
  • Practical architecture: a lightweight AgentAdapter preserves existing agent loops while enabling interchangeable manager/executor/auditor backends, so the approach applies across models and interfaces (GUI, CLI, hybrid).
Who It's For and Tradeoffs

Great fit if you build or evaluate agents that must sustain many dependent steps across changing environments (desktop automation, multi-tool pipelines, terminal workflows). The harness reduces silent failure modes and improves the conversion of partial progress into full task completion. Look elsewhere if your environment cannot support reliable read-only auditing or if the task fundamentally requires a single continuous in-context reasoning trace (where resetting executor context would lose necessary intermediate content). There is added orchestration cost (more round trips and audit work) and reliance on environment observability.

Where It Fits

This paper is most relevant to researchers and practitioners focused on long-horizon agent robustness, benchmark-driven evaluation, and agent orchestration. It complements work on agent planning and tool use by focusing on state hygiene, verification, and bounded execution contexts rather than larger models or new planning algorithms.

Information

  • Websitearxiv.org
  • AuthorsZiyu Ma, Hailang Huang, Shun Zou, Yong Wang, Shidong Yang, Yiming Hu, Fei Wei, XiangXiang Chu
  • Published date2026/08/03

More Items

Introduces physical rank consistency (PRC) and ActionPiece, a learned action tokenizer that preserves local physical relationships via joint supervision of representation learning and quantization, improving autoregressive vision-language-action policy performance across multiple benchmarks.

Extracts replayable behaviors from working web apps and converts them into verifiable, reference-guided software-engineering tasks to evaluate and diagnose coding agents; includes an automated pipeline that scales to thousands of tasks.

Estimates a model's probability of correctness by leveraging its past graded episodes: XConf recalls similar past cases and prompts the model to reflect and restate confidence, improving calibration and discrimination across reasoning, coding, multimodal QA, and agent tasks at much lower generation cost.