Evaluates whether role-playing language agents follow a character's evolving psychological arc rather than a fixed persona, using ArcANE — an automatically constructed benchmark spanning 17 novels and 80 principal characters. Tests both in-text and out-of-text scenarios and compares context strategies and fine-tuned models.
AI Dataset2026
End-to-end evaluation framework for conversational voice agents that runs bot-to-bot audio simulations and scores agents on task accuracy (EVA-A) and interaction experience (EVA-X). Includes per-scenario backend state, accent/noise perturbations, and 213 scenarios across airline, healthcare HR, and enterprise IT domains.
AI Agent Papers2026
Dynamic interactive benchmark that tests whether LLM agents can adaptively plan and re-plan when world and user constraints are progressively revealed. Built on 307 household tasks with a multi-turn protocol that exposes hidden constraints only after plan violations, emphasizing iterative revision and constraint inference.
AI Dataset2026
Provides a complete, lightly-processed export of AI Village's >1-year multi-agent data: per-agent computer sessions (with screenshots), turn-by-turn computer-use logs, group chats, agent memories, goals, and daily summaries for research into agentic behaviour, multi-agent dynamics, long-horizon memory, and AI safety. Access is manually reviewed.
AI Agent Papers2026
Measures how coding agents explore repositories by asking them to return a ranked, line-level list of code regions relevant to an issue under a fixed line budget. Covers 848 issues across 203 repos and 10 languages; evaluates coverage, ranking, and context-efficiency to isolate exploration quality.
AI Video Papers2026
Decouples perception and reasoning for hours-long videos by streaming inputs into a three-tier Hierarchical Graph Memory and using an agentic Observation–Reason–Action retrieval loop; reduces reasoning context to ~2% of full video while improving benchmark accuracy.
AI Agent2026
Provides a virtual, durable filesystem and pluggable execution runtimes for agents running inside Cloudflare Durable Objects. Offers container, isolate-shell, and isolate-JS backends; preview-stage API with ~10GB Durable Object backing and FUSE-mounted container trade-offs.
AI Agent Papers2026
Benchmark for long-horizon computer-use agents that must orchestrate GUI, CLI, and code operations within single trajectories across 114 real-world tasks. Evaluated on a real Ubuntu desktop and paired with a trajectory-aware judge that inspects deliverables, artifacts, and action traces—revealing a top PassRate of ~41.2%.
AI Agent Papers2026
A benchmark that evaluates interactive spatial reasoning for multimodal agents in realistic tasks. It unifies eight heterogeneous simulators under a simulator-agnostic protocol, provides 760 human-annotated tasks with vision-only partial observability, and uses text-based actions plus terminal-state verification to measure task success.
AI Agent Papers2026
Guides LLM-based agents to decompose long-horizon research problems and delegate subtasks to constrained subagents, then fine-tunes models on harness-generated trajectories so delegation decisions become internalized. Reports SearchSwarm-30B-A3B achieving top BrowseComp scores for its scale.
AI Agent Papers2026
Turns raw datasets into verifiable multimodal news features via a multi-agent newsroom pipeline. Key innovations: (1) an Inspector that links each claim to data/code/external references for re-execution and audit; (2) multimodal asset generation (interactive maps, audio, visuals) tailored to the story.
AI Agent Papers2026
Lets a single LLM simultaneously act as agent and environment to bootstrap co-evolutional training — using state-prediction process rewards (World-In-Agent) and failure-mode retrieval (Agent-In-World) to reshape training data; reports ~4% average benchmark gain.