LensVLM introduces a search-and-zoom workflow for long-document understanding: pages are rendered as deterministically compressed images, the model rapidly scans all compressed pages to find likely evidence, then requests full-resolution text or images only for the chosen pages. This avoids loading entire documents into the model context while preserving answer quality for long-context QA tasks.
Key Capabilities
- Selective context expansion: operates in two phases (scan compressed pages → expand selected pages) so only a small subset of pages is decompressed into the model context, reducing token and KV-cache usage by large factors.
- Tunable compression: supports deterministic compression ratios (5x, 10x, 15x) with workflows that trade off latency and storage for context size; reported effective compression ~4.3x while maintaining close-to-full-text accuracy.
- Training recipe: finetuned from Qwen3.5-9B using synthetic supervised trajectories generated by a larger model, followed by reinforcement learning with DAPO to incentivize correct tool use and robust page selection.
- Practical metrics and constraints: the 9B variant learns the search-and-zoom behavior (smaller variants struggle); reported accuracy improvements when using expansion (e.g., ~69% vs ~31% at 5x); latency typically increases due to two-pass operation; source text or high-res pages must be available to perform expansions.
Who it's for and trade-offs
Great fit if you need question answering or evidence retrieval over very long documents but want to limit model context size and KV-cache footprint (e.g., multi-page reports, books, long PDFs). Look elsewhere if you require sub-second single-pass latency, cannot host full-resolution source pages for expansion, or need commercial-use licensing—LensVLM weights are provided under Apple’s Machine Learning Research Model License, which restricts use to non-commercial research.
Where it fits
Architecturally it sits between direct full-text context models and retrieval/RAG approaches: instead of external retrieval or chunking, it compresses the whole document visually and learns a learned-tooled expansion step to get only the necessary high-resolution evidence.
