AIAny
Computer Vision Papers2026
Icon for item

Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs

Decides when a diffusion vision-language model needs to process an image by checking whether text-only predictions agree with its latest image-conditioned predictions. Retains 99.6% of baseline scores across ten benchmarks at 35.3% of decoding FLOPs.

Visit Website

Introduction

Repeatedly processing an image at every denoising step sounds necessary, but the model’s predictions often do not depend on it: text-only and image-conditioned predictions agree for 83.4% of unmasked decisions on average. The key idea here is to turn that redundancy into a per-step certificate, skipping image computation only when the current text-only prediction confirms an earlier image-grounded one.

Key Findings
  • Image dependence varies substantially by task, from 7.9% of steps on MMBench to 43.8% on DocVQA. That makes a fixed image-access schedule a poor fit for many workloads.
  • LOWN reuses image-conditioned predictions when a text-only forward agrees with them, and invokes a full visual forward when agreement is insufficient. The same check determines how many tokens can be unmasked, so it adapts both visual access and decoding progress.
  • Across ten benchmarks, it retains 99.6% of LLaDA-V’s average score while using 35.3% of baseline decoding FLOPs, without extra training or a tuned threshold. It also transfers to other LLaDA-based models and can combine with visual token pruning.
Method and Trade-offs

A recent image-conditioned prediction acts as an anchor. At each step, LOWN checks masked positions with a cheaper text-only forward; agreement certifies a confidence-ordered prefix for reuse. If that prefix is too short, the method recomputes with the image and refreshes the anchor. This is not a guarantee that reused tokens match a fresh full forward: errors can occur and become more likely as the anchor ages. Visual-heavy tasks such as chart and document reading also offer less opportunity to skip image processing.

Who It’s For

Great fit for researchers and practitioners optimizing diffusion VLM inference, especially on tasks where visual inputs frequently leave token decisions unchanged. Look elsewhere if predictable, preset decoding cost or strict agreement with every fresh image-conditioned prediction is essential; LOWN’s savings depend on the response trajectory and its reuse rule permits occasional prediction differences.

Back

Information

  • Websitearxiv.org
  • OrganizationsKAIST, AITRICS
  • AuthorsDonghyeon Cho, Joowon Kim, Seungho Shin, Eunho Yang
  • Published date2026/10/27

Categories

Tags

More Items

AI Agent Papers2026
Icon for item

WorkFactory: Turning Professional Intents into Executable Work for Scalable Agent Training

Ziyang Zeng, Tingyang Chen +11IQuest Research, Zhejiang University +4

Builds executable professional tasks by mapping required information dependencies and repairing gaps revealed during test runs. It produces 112,686 tasks across 612 occupations for collecting teacher demonstrations and training working agents.

paperai-agentdatasetstraining-dataqwen+2
Computer Vision Papers2026
Icon for item

Intrinsic Distribution Matching

Deyuan Liu, Yansen Han +8Inclusion AI, Westlake University +4

Improves few-step text-to-image distillation by matching feature means and per-coordinate standard deviations. It retains DMD’s full score correction and uses the generator’s EMA copy for features, avoiding an external encoder.

paperdistillationai-imagelorabenchmark
Large Language Model Papers2026
Icon for item

Your Benchmark Is an Empirical Measure Over Difficulty

Yifan Sun, Naicheng Yu +3University of Illinois Urbana-Champaign, University of California San Diego +1

Treats a benchmark as a distribution over item difficulty, measuring both mismatch with a target distribution and uncovered difficulty regions. BenchFill rewrites items to fill the largest gaps.

paperbenchmarkbenchmarksevaluationLLM