Category

Metis: Memory Foundation Model

Presents Metis, a prototype memory foundation model that embeds a persistent native memory state into the backbone so historical experience is compressed and accessed via memory attention. Key features: forward-only, gradient-free online memory updates; memory-specific mid-training objectives; and a dual text/code memory design.

Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes

Systematically studies how language and vision interact during unified multimodal pretraining, identifies mechanisms that enable modality synergy versus competition, demonstrates the benefit of early joint training, and derives efficient pretraining recipes validated at scale.