Turns LLMs’ physical knowledge into video training data using simulator constraints, physical verification, and iterative correction. It also repairs physical errors in generated videos and releases 100K videos spanning 2,500 physical events.
Turns screen observations into keyboard and mouse actions to complete long-horizon game tasks. Sparse reasoning, persistent memory, and online reinforcement learning support real-time interaction and multi-step decisions.
Generates videos directly in pixel space without a pretrained autoencoder or cascaded refinement stages. It can jointly produce RGB frames and disparity maps, helping generated scenes follow target camera motion.
Speeds up lossless speculative decoding for vision-language models by focusing drafter training on visually dependent tokens. Sparse attention also reduces redundant work on long video inputs; reported speedups reach 3.58×.
Proposes a framework for robots that combine foundation models with an embodied runtime to interpret intent, call skills, verify outcomes, and adapt. It also outlines an evaluation framework and a roadmap toward sustained autonomy.