Zhongxi Chen, Yi Yang +7Sharpa Robotics, SJTU-Sharpa Joint Lab for Embodied AI, Shanghai Jiao Tong University +2
Transfers hand and scene dynamics from an action-conditioned spatial world model into a dexterous manipulation policy during training. At deployment, the policy uses RGB, language, and robot state without depth input or explicit future prediction.
Lets a frozen model learn robot manipulation from one demonstration by rehearsing in a simulated world and storing feedback in context rather than weights. It reports 83.4% mean success across 50 RoboTwin 2.0 tasks.