Detects and filters spurious token-level teacher supervision in on-policy distillation by estimating input-groundedness and removing high-impact misleading updates, improving OPD on both LLM and VLM benchmarks.
Generates large-scale, explorable 3D open-world scenes from open-ended text prompts, producing editable instance-level assets and a consistent global terrain. Uses agentic planning to convert text into region/terrain/asset specifications and a coarse-to-fine pipeline for terrain construction, mesh reconstruction, and render-based refinement.
Orchestrates reasoning, external tool use, and native image generation under one unified multimodal agent policy via post-training. Introduces RAD-GRPO for agentic reinforcement fine-tuning and releases training data plus the full post-training infrastructure.
A 27B-parameter causal language model with a native vision encoder for image/video+text understanding, long-horizon agentic tasks, and tunable thinking-mode reasoning. Native 262,144-token context (extensible to 1,000,000) and production-focused inference recipes.
Systematically studies how language and vision interact during unified multimodal pretraining, identifies mechanisms that enable modality synergy versus competition, demonstrates the benefit of early joint training, and derives efficient pretraining recipes validated at scale.
Evaluates VLMs' ability to form global spatial awareness from long-horizon egocentric video. Introduces GST-Bench: a VQA benchmark with human-verified questions from 6,790 minutes of synthetic video, reveals a large gap (best zero-shot 42.68 vs human 79.08) and provides GST-Train dataset.
Evaluates schema-guided structured extraction from documents: given a document and a JSON schema, systems must return a schema-valid JSON with page-and-box grounding. Covers 370 documents (4,869 pages) across 8 business domains and 67 document types; scores value accuracy, word/page grounding, and long-list completeness.
A small public sample of egocentric human demonstration video with synchronized 3D hand and body pose annotations for imitation learning and embodied-AI research. Delivered in Parquet and common multimodal packages (LeRobot, MCAP) for schema inspection before requesting gated access to larger EgoSuite releases.
Provides raw, unscripted first-person household video footage for training vision and embodied AI models. Released incrementally on Hugging Face in WebDataset shards with metadata parquets under Apache‑2.0; current raw tier contains ~7,834 hours (≈397k videos).
Lets an AI agent control a real or cloud phone by tapping, typing, scrolling and reading the screen. Works with iPhone via macOS iPhone Mirroring, Android over adb, or rented cloud phones; host-side OCR, accessibility trees and CLI helpers provide verifiable UI actions without jailbreaking. Some manual phone pairing/setup is required.
Turns adapter placement for PEFT on YOLO-family real-time detectors into an auditable constraint-planning problem that emits budgeted target-module plans or calibrated refusals; shows planner-selected RS-LoRA improves mAP and cuts peak training memory in evaluated detectors.
A 29.6B-parameter multimodal causal language model with a dedicated ViT-G/14 perception encoder for running agentic, tool-using, multimodal reasoning locally on consumer hardware. Offers 4-bit quantized weights and a DFlash drafter for speculative decoding to reduce memory and speed up generation.