Proposes chunk-level multimodal retrieval and chunk-adaptive reranking for retrieval-augmented generation on long egocentric videos; introduces V-RAGBench to decouple retrieval vs. generation evaluation and CARVE to run parallel retrievers and select per-chunk configurations.
Provides a training-free, code-as-action framework that lets VLM-backed agents write and run stateful Python cells to compose perception and geometry primitives for open-ended 3D/4D spatial reasoning. Demonstrates consistent gains across 20 benchmarks and multiple VLM backbones.
A quantized 27B coder LLM fine-tuned for repository-level code generation, multi-turn tool calling, and agentic workflows — packaged for local GGUF/llama.cpp deployment with MTP speculative decoding and trace-inversion SFT. Optimized for developer tooling; experimental and not fully safety-validated.
Benchmark for evaluating multimodal LLM safety in Korean cultural contexts — includes KSAFE-MM-G which localizes global safety queries into Korean scenarios and KSAFE-MM-C which targets culture-specific visual-textual vulnerabilities. Provides curated image–text pairs and jailbreak-style prompts to reveal both unsafe behaviors and over-refusal.
Adds interleaved text–image generation to existing image generators via a multi-agent pipeline: a planner sequences stepwise instructions, a critic detects and refines failures, and single-step RL (GRPO) reinforces per-step corrections—suited for visual narratives and embodied guidance.
Provides an open-weight native multimodal agent that understands text and images within a 1,048,576-token context window for long-horizon coding, visual reasoning, and tool-driven workflows. Uses a 2.8T-parameter Mixture-of-Experts architecture (KDA + AttnRes) with MXFP4 quantization; best suited for research and large-scale inference setups.
Learns, maintains, and runs unified world models for Physical AI using a cross-embodiment pretraining curriculum and a hybrid linear temporal-attention architecture. Emphasizes long-horizon state persistence, theoretical bounds on error accumulation, and deployment-aware low-latency inference for real-world embodied agents.
Controllable long-horizon text/image-to-video generation that supports camera navigation, revisits, and promptable events across photorealistic and stylized domains. Introduces camera-aware positional encoding (E-PRoPE), memory-conditioned scene persistence, causal-forcing distillation, and RL alignment to retain camera control and reduce drift.
Language-conditioned robot policy that reuses a pretrained geometric foundation model and inserts a causal future predictor at an intermediate layer so the same backbone produces future 3D-aware features and action outputs, enabling geometry-aware temporal prediction with minimal architectural change.
Converts large-scale egocentric human videos into robot-format pseudo-action trajectories and introduces ACE-EGO-0, a VLA pretraining framework that unifies camera-space actions, morphology conditioning, and reliability-aware weighting to jointly learn from noisy human and high-quality robot data for improved robotic manipulation transfer.
Provides a harness that lets language models control embodied manipulation via iterative perception–reasoning–action loops, semantic action abstractions, and multimodal observations. Demonstrates distilling capabilities into a 4B open-source model with under 2K simulated trajectories and shows sim-to-real generalization.
Proposes ZPPO, a distillation method that keeps the teacher inside prompts rather than injecting teacher gradients, using binary- and negative-candidate prompts plus a prompt replay buffer to recover learning signal on hard examples; shows gains for small Qwen3.5 students across 31 multimodal benchmarks.