AI Agent Papers2026

FrontierPhysics: Evaluating Agents for End-to-End Physics Research

Evaluates AI agents on 56 research-based physics tasks, from literature review and experimentation to manuscript writing. It combines deterministic checks with expert rubrics to assess both correctness and research quality.

Visit Website

Introduction

A physics agent can produce a plausible result and still fail at research: it may overlook a key paper, validate its own mistaken assumption, or miss which experimental error matters. FrontierPhysics makes those gaps visible by evaluating the entire research process, not just the final answer.

Key Findings

The benchmark contains 56 tasks contributed by 42 physicists, spanning nine areas of theoretical, computational, and experimental physics. Tasks draw on real research projects rather than isolated textbook questions.
Agents must review literature, plan and conduct research, produce structured artifacts, and write a scientific manuscript. This tests whether results and their written claims agree.
Evaluation pairs deterministic checks of outputs with expert-authored rubrics for research quality. Across 22 models in 26 configurations, the best reported pass rate was 27.4% (46 of 168 rollouts).
In trajectory reviews, failures were more often associated with insufficiently rigorous reasoning, missing expert intuition, and weak literature synthesis than with basic physics knowledge or execution errors. That shifts attention from what agents know to how they investigate and verify.

Who It’s For—and the Trade-off

Great fit for researchers studying scientific agents, benchmark designers, and anyone asking whether AI can contribute to real physics workflows rather than solve self-contained problems. Look elsewhere if you need a broad measure of general scientific ability: these 56 expert-designed tasks provide a demanding, physics-focused evaluation, not a complete proxy for open-ended research.

Back

Information

Websitearxiv.org
OrganizationsBenchFlow, University of California, Berkeley, California Institute of Technology, Harvard University, University of Washington, Stanford University, Amazon, University of Pittsburgh, University of California, San Diego, Independent Researcher, ZEISS, Georgia Institute of Technology, Lawrence Berkeley National Laboratory, University of Michigan, Ann Arbor, The Scripps Research Institute, Oak Ridge National Laboratory, University of Texas at Austin, Purdue University, University of Toronto, University of Southern California, Ludwig-Maximilians-Universität München, Carnegie Mellon University, University of California, Davis, Princeton University, Northeastern University, University of Pennsylvania, University of Minnesota, Twin Cities, New York University, Duke University, Nanyang Technological University, Pipeline Lab, RLWRLD, University of Palermo, IBM, University of Amsterdam, University of California, Santa Cruz, Prime Intellect
AuthorsBingran You, Wenjun Ke, Ria Rosenauer, Bo Liu, Wenbo Chen, Min Chen, Shuqi Xu, Yue Ma, Jiechao Feng, Bo Chen …
Published date2026/11/04

More Items

AI Agent Papers2026

PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data

Kewei Zhang, Juncheng Ma +12Peking University

Turns a robot calibration episode and data request into synchronized, action-labelled training data: an agent builds the simulated task, while a video model renders realistic observations. Across five robot tasks, farm-trained policies scored 87 successes under unseen appearances, versus 46 for teleoperation-trained policies.

paper robotics embodied-ai manipulation training-data+2

AI Video Papers2026

GameVerse: A Minute-Scale Gameplay Dataset for Long-Horizon Interactive World Modeling

Kang He, Wenshuo Peng +8Alaya Lab, Shanghai Innovation Institute +2

Provides minute-scale continuous gameplay video, camera trajectories, and structured annotations for training interactive world models that remain coherent over longer rollouts. Its two-minute sequences address the shortage of long-horizon training data.

paper datasets video vision world-model+2

AI Dataset2026

Axion 1.0 Training Data

Yuqi Jia, Yunjie Tian +5Duke University, ByteDance +2

Provides scientific-reasoning and tool-use trajectories for training agent foundation models, alongside reinforcement-learning tasks. Its staged data makes the Axion 1.0 training process easier to inspect and reproduce.

datasets training-data sft RL ai-agent+4