Explores unsupervised visual pretraining on visually rich documents to improve language-model intelligence; shows visual-pretrained models outperform text-only counterparts on the same corpora. Key aspects: direct use of images/layouts (no OCR-only pipeline), scalable across backbones and benchmarks.