Evaluates whether MiniMax-H3 can infer latent physical events from incomplete multimodal evidence and produce videos consistent with those inferences. Builds a 517-instance benchmark across four reasoning scenarios (multi-view, audio disambiguation, video continuation, audiovisual integration) and quantifies modality-specific failure modes.
Benchmark for typed probabilistic decisions: given one shared state, a model answers multiple typed questions at once and returns full probability distributions (noul/choice/score). Contains four workflows with train/test splits and soft gold labels from teacher samples, designed to evaluate accuracy, calibration, and latency trade-offs.
Introduces physical rank consistency (PRC) and ActionPiece, a learned action tokenizer that preserves local physical relationships via joint supervision of representation learning and quantization, improving autoregressive vision-language-action policy performance across multiple benchmarks.
Extracts replayable behaviors from working web apps and converts them into verifiable, reference-guided software-engineering tasks to evaluate and diagnose coding agents; includes an automated pipeline that scales to thousands of tasks.
Finds token- and API-cost-saving harness mechanisms for long-horizon coding agents using automated recursive self-improvement; packages four surviving mechanisms (action fusion, context compaction, observation archiving, delegated reading) to cut recorded token traffic ~44.7–49.0% and API cost by about one third while preserving most capability.
Measures how individual harness components—planning, action space, and context management—affect coding agents' success, cost, and behavior. Uses a modular harness across 176 matched settings on SWE-Bench Verified and Terminal-Bench 2.1 to isolate component effects and surface model- and budget-dependent trade-offs.
Provides reproducible, multi‑platform environments and execution‑grounded verification for hybrid computer‑use agents that interleave GUI exploration and code/CLI. Key features: a five‑platform harness, RecreationBench (250 tasks), reference‑grounded programmatic and visual assertions for automatic scoring.
Provides a complete benchmark and training release for native audio‑visual dialogue: 2,800 synthesized audio‑visual dialogues, tiered rubrics, scoring code, ~28GB of 1080p media, and an RL reward recipe to evaluate and train omni models that take video+audio and return text.
Introduces a rubric-based approach for video reward modeling that generates explicit, query-adaptive evaluation criteria before scoring to reduce scalar drift; proposes RGPO, a two-stage training (seed warm-up + joint optimization) that achieves strong pointwise and pairwise evaluation with high data efficiency.
Uses a compact discrete-action interface (translation, rotation, gripper) to let frozen vision–language models control robots in closed-loop, and leverages in-context (one-shot) demonstrations to substantially improve manipulation success without robot-specific training.
Provides an open family of K–12 foundation models and a capability-oriented instruction-tuning corpus that teaches models to solve curriculum problems, map items to curriculum concepts, diagnose learner errors, and produce scaffolded pedagogical responses.
Maps a state and question to typed probabilistic decisions (choice distributions, yes/no probabilities, or scored/ordinal outputs) across controlled synthetic tasks. Offers multiple frozen configs with train/calibration/validation/test/OOD splits, Parquet + raw JSONL exports, and reproducible manifests and provenance.