AIAny
Icon for item

OmniScientist: An Omni-Modal Omni-Discipline AI Scientist

Conducts end-to-end multidisciplinary research directly from heterogeneous raw evidence using lifecycle-wide perception and three autonomous agents (Ideation, Experiment, Writeup). Integrates perceptual analysis, execution provenance, and code-enforced checks to produce executable analyses, validated results, and compiled manuscripts across many modalities.

Introduction

Most automated research systems limit themselves to text, labels, or precomputed summaries; that leaves spatial, temporal, cross-channel and procedural relations—often decisive in scientific work—out of the loop. OmniScientist's core insight is that keeping raw observation and perception available throughout the research lifecycle changes what an AI scientist can ask, test, and claim.

Key Findings
  • Lifecycle design: a perception layer plus three sequential autonomous agents (Ideation → Experiment → Writeup) run inside a deterministic pipeline so observations can reshape hypotheses, experimental choices, and final claims. This enforces numerical traceability and execution provenance via code checks.
  • Empirical evaluation: completed full raw-data→paper workflows on 36 real-data cases spanning 5 discipline families and 12 modalities (images, signals, audio, video, 3-D structures, trajectories, tables, formulae, graphs). Generated papers scored a mean overall paper score of 6.3 (reference backbone).
  • Value of perception: compared to a blind baseline that only saw scalar features, direct perception improved all evaluated dimensions and won 85% of head-to-head judgments, indicating perceptual access materially changes question selection, analyses, and supported claims.
Who it's for and tradeoffs

Great fit if you need an evidence-grounded research pipeline that must reason over spatial/temporal/multi-channel data and produce reproducible execution records and manuscripts. It is particularly relevant for multidisciplinary teams prototyping automated analysis across heterogeneous modalities. Look elsewhere if you need a lightweight prompt-only pipeline (OmniScientist prioritizes perceptual tooling, execution tracing, and code-enforced rigour, which adds system complexity and compute requirements).

Mechanisms (brief)

The system groups evidence into four discipline-independent families (perceptual, symbolic, quantitative-statistical, procedural), exposes perception tools (spatial, temporal, cross-channel, statistical, dynamic), and performs idea/rigour/claim checks in code (novelty screening, statistical validity, anti-HARKing, multiple comparisons, provenance). The result is an end-to-end flow from raw artifacts to executable experiments and a compiled manuscript.

Information

  • Websitearxiv.org
  • Organizations1National University of Singapore 2University of Oxford, Project page: https://omni-scientist.github.io, Software & Skill: https://github.com/Omni-Scientist/OmniScientist
  • AuthorsBobo Li, Hao Fei, Tianjie Ju, Mong-Li Lee, Wynne Hsu
  • Published date2026/08/13

Categories

More Items

Turns natural-language PLC requirements into verified, runnable IEC 61131-3 Structured Text by driving a closed loop of generation, compilation, deployment, and behavioral verification on a live OpenPLC runtime. The verification-gated harness forces inputs, traces execution, repairs failures, and renders ladder diagrams plus process simulation to raise dynamic runtime pass rates.

Enables closed-loop execution for embodied agents by evolving code-based runtime critics and recovery skills online while keeping the base policy frozen. Combines three timescale loops with Z-Infra rollout infrastructure; reports 90.8% on LIBERO-Pro, 93.6% on RoboCasa and an 11.1× inference speedup.

Encodes videos into a Film Knowledge Graph and reconstructs them to learn agent-native, editable video representations for agentic reasoning and manipulation. Uses agentic auto-encoding with dual-loop textual-gradient optimization, reports large reconstruction gains, and releases a benchmark and dataset.