Models visual preference as distributions over rubric scores and introduces Z-Reward, a teacher–student framework that decouples reasoning-heavy judgment (teacher trained with GDSO) from efficient deployment (student via RISD). Demonstrates higher human-preference accuracy and works as a differentiable reward for text-to-image optimization.
Adds discrete audio tokens and an audio encoder to a 30B MoE text LLM so a single model can perform ASR, speech translation, TTS, text-to-audio and speech-to-speech while preserving text reasoning and long-context capabilities; supports thinking/instruct modes and up to 1M-token context.
A benchmark that evaluates interactive spatial reasoning for multimodal agents in realistic tasks. It unifies eight heterogeneous simulators under a simulator-agnostic protocol, provides 760 human-annotated tasks with vision-only partial observability, and uses text-based actions plus terminal-state verification to measure task success.
Turns raw datasets into verifiable multimodal news features via a multi-agent newsroom pipeline. Key innovations: (1) an Inspector that links each claim to data/code/external references for re-execution and audit; (2) multimodal asset generation (interactive maps, audio, visuals) tailored to the story.
End-to-end framework for controlled character animation that transfers motion from driving videos to reference characters without intermediate pose or background representations. Introduces the MotionPair‑60K end-to-end motion-transfer dataset, in‑context mask conditioning and mode‑specific RoPE for task unification, plus Bias‑Aware DPO to mitigate synthetic-detail errors.
Provides 500+ hours of human whole-body teleoperation demonstrations for humanoid robot learning in real homes, with synchronized video, joint states, action traces and language annotations. Includes 23K+ episodes, fine-grained subtask labels, and raw ROS/MCAP plus compressed LeRobot formats.
Generates text from interleaved text, image, and short-video inputs using discrete diffusion and block‑autoregressive multi‑canvas sampling; built on a sparse MoE (8/128) Gemma 4 backbone and optimized for low‑latency inference and very long contexts (up to 256K tokens).
Fine-tuned Hugging Face image-generation model that biases Ideogram-style prompts toward photorealistic outputs. Emphasizes natural lighting and realistic materials to reduce prompt tweaking; license not specified.
Orchestrates teams of sub-agents across text, image, audio and video by modality-aware task decomposition, online sub-agent specialization, and parallel execution; introduces DA-GRPO to train Orchestra-o1-8B and reports a ~10.3% accuracy improvement on the OmniGAIA benchmark.
Continuously watches live video and autonomously decides each second whether to speak, stay silent, or delegate; released together with an 8B vision-first model, time-aligned interaction data, training recipe, and a deployable real-time system. Designed for vision-triggered, low-latency streaming scenarios and evaluated across six real-world streams.
A community-distributed GGUF bundle of Google DeepMind’s DiffusionGemma (26B A4B) with multiple quantization variants for local image-text-to-text inference. Targets experimentation and offline deployment via the DiffusionGemma llama.cpp branch and llama-diffusion-cli; choose quantization for GPU memory vs. fidelity trade-offs.
Encodes and clones camera motion from reference videos to generate multi-shot videos — uses a visual "camera grid" to represent camera parameters, trains on million-scale grid–video pairs, and employs a hierarchical prompt-expansion agent to coordinate camera, subject, and action control for multimodal diffusion models.