AIAny
Icon for item

MMFlash: Faster Speculative Decoding for Vision-Language Models

Speeds up lossless speculative decoding for vision-language models by focusing drafter training on visually dependent tokens. Sparse attention also reduces redundant work on long video inputs; reported speedups reach 3.58×.

Introduction

A multimodal drafter can be trained on images and still fail most often precisely where the image matters. MMFlash finds that draft-token acceptance falls from 46% in the least visually dependent quartile to 22% in the most dependent one, then uses that gap to guide training rather than treating every token equally.

Key Findings
  • Visual dependence is measured by the KL divergence between the target model’s next-token distributions with and without the image. This identifies where visual context changes the prediction, rather than relying on attention maps as a proxy.
  • The training loss upweights visually dependent tokens that remain difficult for the drafter, while accounting for the target’s confidence. This directs training effort toward positions where the image affects a sufficiently settled prediction.
  • For long video inputs, sparse draft-context attention reduces redundant computation while retaining useful visual and textual context.
  • Across multimodal benchmarks, the paper reports lossless decoding speedups of up to 3.58×, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.
Methodological Insight

The key distinction is between a token being image-dependent and a token being predictable. Visual dependence alone does not mean extra training will help: if the target distribution is uncertain, the drafter has no stable choice to imitate. MMFlash combines visual-dependence quantiles with target-distribution entropy so that extra loss weight favors tokens whose predictions are both image-sensitive and confident.

Who It’s For

A good fit for researchers and inference teams evaluating block-based speculative decoding on vision-language models, especially workloads with image-sensitive answers or long video prompts. Look elsewhere if you need a standalone generation model: the drafter is paired with a target model and is designed to accelerate its decoding, not replace it. The reported gains are benchmark-dependent, and the visual-dependence scores require offline computation with the target model.

Information

  • Websitearxiv.org
  • OrganizationsNational University of Singapore
  • AuthorsSicheng Feng, Zigeng Chen, Song Wang, Gongfan Fang, Xinchao Wang
  • Published date2026/11/13

More Items

Generates videos directly in pixel space without a pretrained autoencoder or cascaded refinement stages. It can jointly produce RGB frames and disparity maps, helping generated scenes follow target camera motion.

Reduces Transformer parameter count by repeatedly applying shared blocks, then combines their updates with geometric decay and learned attention. On ClimbMix, it beats standard baselines in validation loss with fewer Transformer parameters.

Turns screen observations into keyboard and mouse actions to complete long-horizon game tasks. Sparse reasoning, persistent memory, and online reinforcement learning support real-time interaction and multi-step decisions.