Develops methods to scale agentic AI for sustained, verifiable execution of complex long-horizon work by expanding executable environments and training coordinated agents with a shared execution harness (AgentOS) to maintain state, provenance, and failure recovery.
Evaluates AI agents' ability to complete end-to-end scientific workflows by releasing and assessing 97 tasks from a 300-task FrontierChallenge suite across chemistry, materials, life science, and electrochemistry. Finds that top agent configurations achieved only a 20.6% pass rate despite high partial scores, revealing a gap between partial progress/confident completion claims and actual complete scientific deliverables.
A natively multimodal model for text and image→text generation, long-context reasoning, and complex coding/agent workloads. Uses 320B total / 18B active params with a hybrid sparse+linear attention and manifold-constrained hyper-connections to reduce long-context serving cost.
Converts image-level rewards into explicit intermediate targets for diffusion-model denoising via an on-policy self-distillation loop. Constructs bounded positive/negative targets around anchors from reward gradients, fits those targets with finite updates, and refreshes a behavior policy by EMA—improving aligned performance across backbones while reducing GPU hours.
Evaluates retrieval-augmented generation by decomposing user queries into sub-queries and answers into atomic claims, scoring retrieval by query coverage and generation by claim verifiability. Reference-free benchmark with 800 queries, inlined retrieved chunks, and answers from multiple RAG systems; runs locally without API keys.
Provides aggregated, privacy-preserving cluster outputs from three external research teams' analyses of ~250k Claude/Claude Code conversations; includes per-team CSVs (Stanford, Oxford, METR) for studying human–AI collaboration and model behavior without raw conversations.
Synthesizes, repairs, and self-evolves task-adaptive agent harnesses on demand for off-the-shelf LLM agents, using a trainable harness-intelligence model that distills signals from past configurations. Demonstrates consistent performance gains across benchmarks and model families by producing four-module, composable harnesses.
Analyzes how to generate useful interaction data for LLM agents and proposes the ACE lens — Accuracy, Complexity, divErsity — while factorizing agentic data as (E, q, τ, v). Surveys verification, difficulty calibration, and coverage strategies and outlines implications for training and benchmarks.
A zero-data self-evolution framework that co-trains a Challenger, Solver, and Judge so LLMs can iteratively improve on both verifiable and unverifiable tasks without human labels. Uses role-asymmetry and subtask-amplification preference pairs to train the Judge and sustain improvement.
Measures whether video generators reproduce the correct distribution of possible physical behaviors under repeated rollouts. Introduces PAWBench and PAWEval to convert repeated generations into outcome-level empirical distributions and quantify probabilistic alignment; evaluates 50 scenarios and 11 models and finds no model consistently matches reference probabilities.
Provides a real-scale 3D Hong Kong sandbox to evaluate whether multimodal LLM agents can turn local street-view perception into sustained spatial action, supporting closed-loop first-person interaction, an interactive map, and controlled tests of grounding, long-range navigation, and robustness.
Decides when past post-training updates should be reused for autonomous LLM adaptation by introducing Boundary-Calibrated Intervention Transfer (BCIT). BCIT binds effects to source context, checks applicability and hard conflicts, and runs bounded trials to obtain current-state evidence—reducing harmful updates and improving equal-budget final-model quality.