Personalizes subject-driven videos to preserve human identity and accurate human–object interactions by integrating multimodal references and MLLM-derived semantics. Introduces global multimodal guidance in self-attention and modality-reference embeddings to align MLLM features with VAE tokens, supporting both inter- and intra-subject inputs (e.g., OCR, multi-view).
Evaluates atomic visual perception of multimodal LLMs using 3,000 short visual questions that isolate ten perceptual skills. Built from an error taxonomy across 42 benchmarks, capability-balanced and accompanied by a model leaderboard.
Delivers image and video understanding plus a built-in event‑gated streaming gate — a unified 4B multimodal foundation model that uses codec-aligned tokenization to cut visual tokens by >75% and yield up to 3.5× wall‑clock inference speedup for streaming and long‑horizon video tasks.
Consolidated dataset of detection, visual grounding and pointing annotations with indexed WebDataset image shards and Megatron‑Energon training metadata. Covers diverse visual domains (COCO, RefCOCO, driving, GUI, documents) and uses a normalized spatial grid for cross‑domain vision–language grounding training.
Provides 1,080,814 images extracted from ~65,000 digitised British Library book volumes (c.1510–c.1900), split into four algorithmic image-type configs and packaged as parquet for image–text multimodal research and retrieval.
Evaluates schema-guided structured extraction from documents: given a document and a JSON schema, systems must return a schema-valid JSON with page-and-box grounding. Covers 370 documents (4,869 pages) across 8 business domains and 67 document types; scores value accuracy, word/page grounding, and long-list completeness.
Provides OCR full text for 11.55M public-domain German newspaper pages (1638–1964) with per-page IIIF scans and ALTO XML coordinates; suited for historical NLP, language-model training, and OCR research. Pages carry explicit per-page public-domain licenses.
Multimodal vision-language model optimized for on-device image+text tasks: image captioning, full-page OCR with layout annotation, grounding/bounding-box prediction, and function calling. Built on the LFM2.5-2.6B backbone with a SigLIP2 NaFlex 400M vision encoder and tuned for low-latency, low-memory edge inference.
Provides 115,293 illustrated page images and a 975,345-row manifest sampled from scanned Encyclopaedia Britannica volumes (1768–1929), with per-page classifier probabilities for illustration — ready for image-classification, OCR-aware vision research, and illustration mining.
Compresses conversational histories and long documents into short sequences of continuous soft memory tokens that a frozen decoder can read directly without text reconstruction. Uses a small reader-matched writer that trains only a tiny adapter, achieving 4–16× compression and much faster write/read latencies.