AI Video Papers2026
Haoyu Zhao, Zihao Zhao +11Affiliation: National University of Singapore, Affiliation: Fudan University +1
Evaluates whether MiniMax-H3 can infer latent physical events from incomplete multimodal evidence and produce videos consistent with those inferences. Builds a 517-instance benchmark across four reasoning scenarios (multi-view, audio disambiguation, video continuation, audiovisual integration) and quantifies modality-specific failure modes.
AI Dataset2026
Provides transcripts, translations, and multi-task text labels for speech across 48 languages and about 463,000 hours. The release contains manifests only; audio must be obtained from the original data providers.
AI Dataset2026
Provides a complete benchmark and training release for native audio‑visual dialogue: 2,800 synthesized audio‑visual dialogues, tiered rubrics, scoring code, ~28GB of 1080p media, and an RL reward recipe to evaluate and train omni models that take video+audio and return text.
AI Model2026
Converts multimodal inputs (text, image, audio, video) plus a question and options into calibrated probability distributions over choices. Built on Gemma 4 12B with a 30,000-question fine-tune, optimized for per-question decision classification and low-latency inference (Apache-2.0).
AI Agent Papers2026
Ruixun Liu, Yuxuan Wang +11School of Intelligence Science and Technology, Peking University, State Key Laboratory of General Artificial Intelligence, Peking University +2
Provides a unified benchmark and a spatially aware omni-modal model for spatial audio-visual reasoning in embodied agents — OmniEchoBench (197 scenes, 2,972 QA pairs; 900 FOA-based navigation tasks) plus OmniEcho, which adds an FOA spatial encoder to a pretrained semantic audio pathway to enable spatial QA and sound-guided navigation.
AI Model2026
A sparse-MoE multimodal LLM fine-tuned with large-scale mixed reinforcement learning, supporting text/image/video/audio and a 1M-token context window; designed for long-horizon agentic tasks, coding and visual reasoning with groupwise grading and on-policy distillation.
AI Audio2026
Performs low-latency streaming speech-to-text, emitting one token per selectable 80/120/160 ms clock with configurable transcription delay and a 30s rolling KV cache for unlimited 24/7 transcription. Bilingual (zh/en) and includes semantic VAD.
AI Model2026
Open-weights multimodal LLM checkpoint designed for 1M-token long-context agent runs that combines large-scale on-policy RL with groupwise grading for iterative self-improvement. Key traits: sparse MoE backbone (≈1.02T total / 42B active), text-only output with image/video/audio inputs, MIT-licensed weights on HuggingFace—suited for long-horizon agents and research at significant infra cost.
AI Audio2026
Performs speaker diarization (who spoke when) for live and recorded audio using an open-weight, 100M-parameter streaming-capable model that supports up to eight anonymous speaker channels, overlapping speech, chunked processing, and configurable latency for ASR integration.
AI Video2026
Enables native joint video-and-audio generation and training up to 2K using a dynamic block-sparse attention scheme that cuts attention cost while preserving audio–visual coupling; provides preview checkpoints, training and inference code. High-resolution training requires large multi-GPU (80GB) clusters and CUDA ≥ 12.4.
Evaluates multimodal memory in large audio language models by testing four acoustic evidence types and four memory operations across multi-session spoken histories, stratified across 8K–64K token context budgets.
AI Video Papers2026
Songlin Yang, Xiaotong Zhao +6MMLab@HKUST, The Hong Kong University of Science and Technology, Tencent Video +1
Proposes a forward-process RL method that dynamically localizes gradient updates and adapts multi-reward coordination for joint audio–video diffusion models. Uses bidirectional cross-attention responses for token/layer routing and preference-preserving reweighting to balance competing objectives, improving modality quality, alignment, and synchronization.