Multilingual scene text recognition is bottlenecked by scarce real data for most languages and by solutions that either replicate per-language recognizers or rely on very large vision-language models. The paper exploits the empirical prior that a text instance typically contains a single script, turning script routing into a low-ambiguity decision and enabling a sparse, expert-based decoder that activates only a few script-aligned experts per image.
Key Findings
- ScriptMoE architecture: shares a single visual encoder and replaces the dense decoder with a sparse MoE block that includes an image-level router, a shared cross-script expert, and script-aligned experts; the router dispatches each image to the top-2 script experts plus the shared expert, limiting interference across scripts. This reduces per-language parameter bloat while keeping specialization.
- TextMuSS-10M dataset: a large synthetic scene-text corpus spanning 10 scripts and 229 languages to provide balanced supervision where real data is lacking.
- Empirical results: ScriptMoE achieves 82.06% average accuracy on the authors’ TextMuSS-Bench (10 scripts, 10,899 images), outperforming the strongest STR baselines (by ~1.31% average and 2–3 points on Arabic, Thai, Tibetan). Replacing only the recognizer in PP-OCRv5 raises end-to-end CC-OCR F1 from 65.71% to 80.89%, slightly exceeding a best VLM baseline while using far fewer parameters.
- Practical implication: the sparse expert decoder yields a favorable tradeoff between specialization and model size, making it attractive for multilingual OCR deployments that must cover many scripts without per-language models.
Who it's for and trade-offs
Great fit if you build or research multilingual OCR systems that must scale across many scripts or low-resource languages and want better per-script accuracy without exploding model size. It benefits settings where script detection is feasible at the image level. Look elsewhere if you need full multimodal language understanding beyond transcription (e.g., semantic VLM tasks) or if you cannot accept synthetic-data-induced biases without substantial real-data fine-tuning. The approach requires script-aligned expert design and high-quality synthetic generation to realize its gains.