Systematically studies how language and vision interact during unified multimodal pretraining, identifies mechanisms that enable modality synergy versus competition, demonstrates the benefit of early joint training, and derives efficient pretraining recipes validated at scale.
AI Video Papers2026
Evaluates VLMs' ability to form global spatial awareness from long-horizon egocentric video. Introduces GST-Bench: a VQA benchmark with human-verified questions from 6,790 minutes of synthetic video, reveals a large gap (best zero-shot 42.68 vs human 79.08) and provides GST-Train dataset.
AI Dataset2026
Provides 1,080,814 images extracted from ~65,000 digitised British Library book volumes (c.1510–c.1900), split into four algorithmic image-type configs and packaged as parquet for image–text multimodal research and retrieval.
Generates retrieval-centric Chain-of-Thought (RC-CoT) over initially retrieved candidates to improve unified multimodal retrieval via reranking or full-corpus re-retrieval with a dual-mode embedder. Trains an embedder–adviser framework (UniME-R1) using mined hard negatives, supervised learning, and retrieval-oriented reinforcement learning.
AI Video2026
Turns a short prompt plus aspect ratio and duration into a structured, shot-by-shot audio-video description for text-to-audio-video generation. A PEFT LoRA on Qwen3.6-27B that expands timing, camera motion, continuity, and synchronized diegetic/non‑diegetic sound; text-only and requires MiniMax-H3 + LightX2V to produce final AV.
AI Dataset2026
10,000-hour head-and-wrist egocentric dataset pairing synchronized head and wrist video with left/right 3D hand pose and optional full-body pose; provided in LeRobot/MCAP formats with episode-level semantic annotations and automated de-identification.
Tao Feng, Fangxu Yu +10University of Illinois Urbana-Champaign, University of Maryland, College Park +3
Frames LLM routing as a sequential decision process and introduces LLMRouter plus the xRouteBench benchmark to develop, evaluate, and deploy learned routing policies across heterogeneous LLMs, optimizing response quality versus inference cost.
AI Dataset2026
A small public sample of egocentric human demonstration video with synchronized 3D hand and body pose annotations for imitation learning and embodied-AI research. Delivered in Parquet and common multimodal packages (LeRobot, MCAP) for schema inspection before requesting gated access to larger EgoSuite releases.
AI Video2026
Packaged diffusers checkpoint of MiniMax H3 for image/text-to-short-video generation with native stereo audio; provided for direct use in diffusers image-to-video pipelines and aimed at easy integration into prototyping and production workflows.
AI Video2026
Provides ComfyUI-compatible conversions and LoRA adapters of the MiniMax‑H3 video+audio generative model, with example presets and demo videos to run short stereo audio+video inference inside ComfyUI workflows.
AI Dataset2026
Provides raw, unscripted first-person household video footage for training vision and embodied AI models. Released incrementally on Hugging Face in WebDataset shards with metadata parquets under Apache‑2.0; current raw tier contains ~7,834 hours (≈397k videos).
AI Dataset2026
Provides 90,000 hours of head-mounted egocentric video paired with synchronized 3D hand pose and an optional 3D full‑body pose add-on, with event-level semantic labels available as a complimentary layer — designed for embodied AI and robotics training at scale.