Motus2: A Self-Evolving General World Model for Dexterous Manipulation
General-purpose dexterous manipulation requires models not only to acquire transferable experience from large-scale interaction data, but also to retain task-relevant information under hand-induced self-occlusion, infer contact states that are difficult to determine visually, and continue improving their policies beyond supervised imitation. However, existing robotic foundation models rely primarily on costly robot demonstrations for action learning. This dependence limits the scalable acquisition of interaction experience and provides no direct mechanism for judging actions from their execution outcomes and subsequently improving the policy.
To address these limitations, Motus2 introduces a self-evolving general world model for dexterous manipulation [5]. Through egocentric pretraining that progresses from monocular to synchronized stereo data, followed by robot-domain adaptation, Motus2 transfers large-scale human interaction experience to robot control. A policy, simulator, and evaluator backed by shared parameters predict and assess the consequences of candidate actions, while model-based reinforcement learning converts value feedback into policy updates. Motus2 further incorporates working memory and a tactile expert to improve historical information retention in long-horizon tasks and enhance contact perception.
Real-World Demonstrations
In the Screw Bulb task, the robot must use its right hand to align a light bulb with the socket and continuously rotate it until the bulb lights up, after which it releases the bulb. This process requires precise spatial alignment, stable grasping, and sustained contact control.
In the Multi-Finger task, the robot must perform a multi-finger grasp on objects of varying shapes and place the target object into a designated container. This task evaluates multi-finger coordination and fine-grained manipulation with a high-degree-of-freedom dexterous hand.
In the Find Square task, the target square is hidden under one of three cups, whose positions are subsequently shuffled. The robot must retain the visual information observed before occlusion and use it to uncover the correct cup, demonstrating the importance of long-horizon visual context in partially observable tasks.
In the Tear Paper task, the robot uses coordinated bimanual control to tear a sheet from a roll of paper. Because the contact and force states involved in tearing are difficult to infer accurately from vision alone, this task primarily evaluates bimanual coordination supported by tactile feedback.
Figure 1: The nine real-world evaluation tasks used for Motus2. Each row shows eight sampled frames from a representative execution.
Technical Approach
Motus2 scales dexterous manipulation along two axes: data scaling through a hierarchical pyramid of human egocentric data, and model scaling through a general world model with an explicit policy-improvement loop.
Model Scaling: A Self-Evolving Closed Loop
Through UniDiffuser-style joint video–action modeling, Motus unified a policy interface, implemented as a world–action model, and a simulator interface, implemented as an action-conditioned world model, within a single shared model [1,4]. Predicting the consequence of an action, however, does not indicate whether that outcome advances the task. Motus2 therefore introduces a third control interface implemented by a value model—the evaluator—which assesses predicted outcomes and provides signals for policy improvement.
Figure 2: Motus2 provides policy, simulator, and evaluator interfaces through a single model with shared parameters.
The three interfaces correspond to different conditional queries over the same parameters. The policy proposes executable action chunks, the simulator predicts their future visual outcomes, and the evaluator estimates the task progress associated with each predicted outcome. These interfaces follow the causal order “context → action → predicted future → value.” An action-first attention mechanism ensures that action tokens cannot access future video or value tokens within the current chunk, while future video tokens can attend to actions and value tokens can attend to both. This design prevents future information from leaking into action prediction. Standard control requires only the policy interface; future video and value generation are invoked only for planning or policy optimization.
Figure 3: Joint pretraining uses bidirectional video–action interaction, whereas robot-domain mid-training and post-training use an action-first mask.
The evaluator is supervised by task progress, with positive and negative targets assigned at the trajectory level. Segments from successful trajectories receive positive targets according to their relative progress, whereas failed teleoperation trajectories and task-irrelevant interactions receive negative targets. The same partition also determines action supervision: only filtered successful trajectories activate action learning, while failed and suboptimal executions are used exclusively to train the simulator and evaluator, for which their recorded actions serve as valid conditioning variables. The model can therefore learn dynamics and outcome evaluation from failures without being trained to reproduce unsuccessful actions.
These interfaces support both test-time scaling and a self-evolving closed loop:
- During test-time Best-of-$N$ planning, the policy proposes multiple candidate action chunks, the simulator predicts the future associated with each candidate, and the evaluator ranks the resulting branches. The robot executes the highest-value branch and replans after receiving a new real-world observation.
- During post-training, DiffusionNFT [3] converts candidate-action value estimates into policy updates, increasing the probability of generating high-value actions while suppressing low-value actions.
Test-time planning improves only the selection among the current candidate actions and does not modify model parameters. DiffusionNFT, by contrast, feeds model-predicted consequences and value estimates back into the action policy. Policy improvement therefore no longer depends exclusively on supervised imitation, closing the loop between decision-making and learning.
Figure 4: Value signals are used for policy updates during post-training and for candidate-action selection during inference.
Data Scaling: From Human Experience to Robot Execution
Model scaling concerns how experience can be used to improve the policy, whereas data scaling concerns how interaction experience can be acquired at scale. Real-robot data provide direct supervision for control but are difficult to collect in large quantities. Human egocentric data are substantially easier to obtain but cannot be translated directly into robot actions. Motus2 establishes a progressive transfer pathway from human interaction to robot execution, enabling large-scale human experience to inform robot control.
This pathway begins with monocular egocentric video. Monocular data cover diverse tasks, objects, and environments, providing visual, semantic, and hand–object interaction priors. Synchronized stereo video and human actions are then introduced. The depth cues and more accurate three-dimensional hand poses provided by stereo views transform purely visual experience into action-associated interaction experience. Finally, robot-domain adaptation uses robot trajectories and human–robot alignment data to ground human interaction priors in an executable robot action space [2].
Figure 5: A hierarchical data pyramid spanning monocular and stereo egocentric data, human–robot alignment data, and robot interaction data.
To support this pathway, Motus2 constructs an egocentric corpus comprising approximately 130,000 raw recording hours and uses more than 100 hours of robot trajectories together with complementary human–robot alignment data during robot-domain mid-training.
Motus2 further evaluates the scaling potential of this pathway using nested stereo subsets ranging from 2k to 20k raw hours. As the amount of data increases, the best held-out action-prediction error at each scale consistently decreases and exhibits an approximately linear relationship with the logarithm of data scale.
Figure 6: Within the data range measured in the paper, increasing the amount of synchronized stereo human data consistently reduces held-out action-prediction error.
Working Memory and Tactile Expert
Beyond model and data scaling, dexterous manipulation presents two additional challenges. First, hand-induced self-occlusion creates partial observability: task-relevant information available early in an episode may no longer be visible when it is needed for a later decision. Second, visual observations are ambiguous with respect to contact state; critical events such as fingertip slip, grasp formation, and release are difficult to infer accurately from images alone. Motus2 addresses these challenges with working memory and a tactile expert, respectively, strengthening temporal context modeling and contact perception.
For partial observability in long-horizon tasks, Motus2 uses a bounded sliding window as its default context, retaining recent observations with fixed cache capacity and per-step computational cost. Motus2 further investigates two long-history mechanisms. Global autoregression retains the complete observation history, but its computational cost grows with task duration. Hybrid memory follows the MemoryWAM [6] design: it preserves the initial anchor frames and recent observations in full while compressing earlier intermediate history into persistent memory tokens.
To resolve contact uncertainty that vision alone cannot eliminate, Motus2 introduces a lightweight tactile expert that uses high-frequency tactile signals to refine actions immediately before execution, without rerunning the full backbone. The main model first denoises the complete action chunk to an intermediate state. Before each short action sub-chunk is executed, the tactile expert reads the latest tactile window and performs the final update. During training, future force prediction is introduced as an auxiliary objective to capture the relationship between actions and changes in contact. During deployment, the module outputs only the actions refined with tactile information.
Experimental Results
The Motus2 experiments evaluate egocentric pretraining and robot-domain adaptation, value-guided policy improvement, long-horizon context mechanisms, and tactile feedback. All real-world experiments use identical initial configurations and a unified success criterion, with 20 trials per method for each task.
Can egocentric experience transfer to robot control? Across five real-world tasks—Place Ball, Multi-Finger, Attach Eraser, Screw Bulb, and Put Phone—WAN-SFT, initialized directly from the video foundation model, achieves a macro-average success rate of 0%. After monocular and stereo egocentric pretraining, Pretrain-SFT reaches 51%. Following additional robot-domain mid-training, Motus2 reaches 84%. These results show that egocentric pretraining and robot-domain adaptation play distinct but successive roles in the data-scaling pathway.
Figure 7: Main experimental results.
Can value feedback improve the policy itself? On Put Phone and Multi-Finger, the base model achieves a macro-average success rate of 65.0%. Best-of-$N$ planning alone increases the success rate to 67.5%. After model-based policy improvement with DiffusionNFT, direct inference reaches 72.5%. Combining policy improvement with planning further raises performance to 75.0%. Compared with test-time planning, which changes only candidate selection, DiffusionNFT yields a larger improvement by updating the policy itself.
Figure 8: Motus2 policy-improvement and planning results.
Can long-term history mitigate partial observability? On the Find Square and Press Button memory tasks, global autoregression achieves a macro-average success rate of 78% in simulation, compared with 52% for hybrid memory. In real-world evaluation, the two methods achieve 57.5% and 25.0%, respectively. These results indicate that, on the evaluated long-horizon tasks, retaining the complete visual history yields better performance, although its computational cost grows with task duration.
Figure 9: Motus2 long-horizon context results.
Can tactile feedback improve contact-intensive manipulation? On the real-world Pull Out the Paper Cup and Tear Paper tasks, introducing the tactile expert increases the macro-average success rate from 60.0% to 72.5%. Performance on Pull Out the Paper Cup increases from 65% to 75%, while performance on Tear Paper increases from 55% to 70%. These results show that tactile feedback improves contact perception and action refinement on the evaluated tasks.
Figure 10: Motus2 tactile-feedback ablation results.
References
[1] Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei Ma, Hang Su, and Jun Zhu. Motus: A Unified Latent Action World Model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 35101–35113, June 2026.
[2] Hongzhe Bi, Lingxuan Wu, Tianwei Lin, Hengkai Tan, Zhizhong Su, Hang Su, and Jun Zhu. H-RDT: Human Manipulation Enhanced Bimanual Robotic Manipulation. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026.
[3] Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, and Ming-Yu Liu. DiffusionNFT: Online Diffusion Reinforcement with Forward Process. In International Conference on Learning Representations, 2026. Oral.
[4] Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, and Jun Zhu. One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale. In International Conference on Machine Learning, pages 1692–1717, 2023.
[5] Jun Zhu, Hengkai Tan, Jintao Zhang, Min Zhao, Fan Bao, and Bo Zhang. General World Models from First-Principles. Manuscript, 2026.
[6] Sizhe Yang, Juncheng Mu, Tianming Wei, Chenhao Lu, Xiaofan Li, Linning Xu, Zhengrong Xue, Zhecheng Yuan, Dahua Lin, Jiangmiao Pang, and Huazhe Xu. MemoryWAM: Efficient World Action Modeling with Persistent Memory. arXiv preprint arXiv:2606.20562, 2026.