Continuously watches live video and autonomously decides each second whether to speak, stay silent, or delegate; released together with an 8B vision-first model, time-aligned interaction data, training recipe, and a deployable real-time system. Designed for vision-triggered, low-latency streaming scenarios and evaluated across six real-world streams.
Encodes and clones camera motion from reference videos to generate multi-shot videos — uses a visual "camera grid" to represent camera parameters, trains on million-scale grid–video pairs, and employs a hierarchical prompt-expansion agent to coordinate camera, subject, and action control for multimodal diffusion models.
Proposes chunk-level multimodal retrieval and chunk-adaptive reranking for retrieval-augmented generation on long egocentric videos; introduces V-RAGBench to decouple retrieval vs. generation evaluation and CARVE to run parallel retrievers and select per-chunk configurations.
Provides 1000+ hours of high-precision optical motion-capture for humanoid robotics and embodied AI, including full-body skeleton, 20+DoF hands, object 6D, and multi-view video at 120 Hz. Sub-mm spatial accuracy, BVH/CSV/NPZ outputs and Unitree G1 retargets; ideal for imitation learning and sim-to-real, with some raw captures gated by license.
Provides experimental GGUF-format quantized weights for MiniMax-M3 to run local multimodal (image‑text‑video) inference via llama.cpp or Unsloth Studio. The model is very large (~428B params) and requires GPU offload or large CPU RAM; llama.cpp currently falls back from sparse to dense attention.
Controllable long-horizon text/image-to-video generation that supports camera navigation, revisits, and promptable events across photorealistic and stylized domains. Introduces camera-aware positional encoding (E-PRoPE), memory-conditioned scene persistence, causal-forcing distillation, and RL alignment to retain camera control and reduce drift.
Language-conditioned robot policy that reuses a pretrained geometric foundation model and inserts a causal future predictor at an intermediate layer so the same backbone produces future 3D-aware features and action outputs, enabling geometry-aware temporal prediction with minimal architectural change.
Converts large-scale egocentric human videos into robot-format pseudo-action trajectories and introduces ACE-EGO-0, a VLA pretraining framework that unifies camera-space actions, morphology conditioning, and reliability-aware weighting to jointly learn from noisy human and high-quality robot data for improved robotic manipulation transfer.
Provides 130k+ bimanual teleoperation trajectories for robot imitation learning, recorded on low-cost YAM two-arm rigs and shared as MCAP episodes with subtask annotations, training code, and checkpoints.
Serves interactive, long-lived streaming video-generation sessions by jointly scheduling session placement and GPU autoscaling to meet tight per-chunk latency. Combines migration-aware placement, load-driven autoscaling, coalesced chunk processing, GPU–CPU offloading and NCCL GPU–GPU migration; reports ~37% reductions in worst-case per-chunk latency and GPU operating cost.
Provides a small, manually annotated benchmark for evaluating vision–language models that convert robot and egocentric manipulation videos into timestamped subtask segments and concise action labels. Contains 100 episodes, 743 gold segments, and MP4 bytes embedded per row.
Provides multiview synthetic RGB video clips with per-frame depth, instance masks, dense long-range 3D point tracks, camera poses, and SMPL‑X human pose/shape labels for 4D reconstruction, tracking, and geometry-aware novel-view synthesis. Includes ~4.7K clips (1.4M frames) and is licensed for AI training.