A multimodal drafter can be trained on images and still fail most often precisely where the image matters. MMFlash finds that draft-token acceptance falls from 46% in the least visually dependent quartile to 22% in the most dependent one, then uses that gap to guide training rather than treating every token equally.
Key Findings
- Visual dependence is measured by the KL divergence between the target model’s next-token distributions with and without the image. This identifies where visual context changes the prediction, rather than relying on attention maps as a proxy.
- The training loss upweights visually dependent tokens that remain difficult for the drafter, while accounting for the target’s confidence. This directs training effort toward positions where the image affects a sufficiently settled prediction.
- For long video inputs, sparse draft-context attention reduces redundant computation while retaining useful visual and textual context.
- Across multimodal benchmarks, the paper reports lossless decoding speedups of up to 3.58×, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.
Methodological Insight
The key distinction is between a token being image-dependent and a token being predictable. Visual dependence alone does not mean extra training will help: if the target distribution is uncertain, the drafter has no stable choice to imitate. MMFlash combines visual-dependence quantiles with target-distribution entropy so that extra loss weight favors tokens whose predictions are both image-sensitive and confident.
Who It’s For
A good fit for researchers and inference teams evaluating block-based speculative decoding on vision-language models, especially workloads with image-sensitive answers or long video prompts. Look elsewhere if you need a standalone generation model: the drafter is paired with a target model and is designed to accelerate its decoding, not replace it. The reported gains are benchmark-dependent, and the visual-dependence scores require offline computation with the target model.