Many hard visual-reasoning tasks fail not because models lack capacity but because evidence needed for grounded answers is scattered across tools and transient by default. OpenVLHarness reframes multimodal reasoning as an iterative, tool-driven process: capture tool outputs as model-readable artifacts, retain them in a persistent multimodal memory, and run small programs over stored evidence to compose final answers.
Key Findings
- Average gains across 22 benchmarks: +9.7–12.6 absolute points. So what: consistent, cross-backbone improvements show tool integration matters more than merely increasing inference budget.
- Representative improvements: +11.2 (Qwen3-VL-8B), +9.7 (Kimi K3), +9.9 (GPT-6 Astra). So what: benefits apply to both open and proprietary MLLMs.
- Efficiency: matches or exceeds stronger baselines while using at most ~37% of their model-token cost. So what: practical for constrained-cost deployments where heavy model-only strategies are expensive.
- Architectural ingredients: a capability layer that normalizes heterogeneous tool outputs into model-readable evidence, persistent multimodal memory to keep images/structured data/web refs addressable, and a coding interface to compute over stored artifacts. So what: decouples perception backends from reasoning and enables reusable evidence across steps.
Who it's for and tradeoffs
Great fit if you evaluate or build multimodal agents that require grounded perception, measurements, or web evidence across multi-step workflows—particularly for research or benchmarks comparing tool-enabled reasoning. Look elsewhere if your use case is single-shot captioning or end-to-end fine-tuned pipelines where adding an external harness complicates latency or deployment constraints. The harness improves reasoning without additional model training, but it introduces engineering complexity (tool integration, memory management) compared to a pure-model approach.
Where it fits
Think of OpenVLHarness as an agent-harness layer that systematically converts diverse tool outputs into durable, composable evidence for MLLMs—an approach that often outperforms simply raising model inference effort on hard multimodal tasks.