WAIC 2026 Special Dialogue | CCTV Interview with Zhu Jun: From Generating Video to Understanding the World

During the World Artificial Intelligence Conference (WAIC), CCTV’s Tech Zero Distance program launched a special series — “Dialogues with 100 Future-Oriented Tech Figures” — inviting Shengshu founder and Deputy Director of Tsinghua University’s Institute for AI, Zhu Jun, for an in-depth conversation with CCTV reporter Zhang Peiling.

From real-time video generation to general-purpose robot brains, from the data value of massive video to the technical pathway of world models, Zhu Jun systematically shared Shengshu’s long-term vision and practice around general world models. Video generation, real-time interaction, and robotic action are not separate technical directions, but a continuous evolution of the general world model — from understanding and prediction to action.

In the interview, Zhu Jun first introduced Shengshu’s newly released real-time interactive model, Vidu S1.

Unlike traditional video models that generate fixed clips in one pass, Vidu S1 continuously incorporates user input into the generation process. Users can interact directly with digital characters through voice, and the model understands emotions and instructions in real time, generating corresponding expressions, movements, and video content.

Vidu S1 supports unlimited-duration continuous generation. Users can also customize character appearance, voice, and persona, creating personalized real-time interactive characters for live-streaming, gaming, digital companionship, and other scenarios.

In Zhu Jun’s view, real-time interaction is not a separate direction from video generation, but an important step toward world models.

Video is not simply a sequence of pixels; it is a record of dynamic changes in the real world. People, objects, environments, actions, and events are all embedded as spatiotemporal changes. Through large-scale video training, models not only generate images but also gradually learn how objects move, how environments change, and what outcomes different actions may produce.

“From video generation to real-time interaction to action generation — these three things are unified at the foundational level,” Zhu Jun said. The model can generate content in the digital world, continuously update its state based on external input, and further generate executable instructions for robots in the physical world.

To outsiders, Shengshu’s move from video foundation models into robotics may seem like a leap. But Zhu Jun says the company’s exploration of world models began early.

In 2024, the team observed unexpectedly strong video dynamics and consistency in the Vidu model, leading them to realize that the spatiotemporal modeling and dynamic prediction capabilities developed for video models could extend beyond content generation into the physical world.

In July 2025, Shengshu released Vidar. The model first predicts future states through video prediction, then uses an inverse dynamics module to map those states into robot actions — providing initial validation that video generation models can support physical action.

In December 2025, Shengshu further released and open-sourced Motus, integrating environmental understanding, future state prediction, and action generation within a unified model framework, exploring the full loop of “understanding — imagination — action.”

In April 2026, Shengshu launched the world action model Motubrain, targeting complex long-horizon tasks and cross-embodiment generalization. Faced with tasks like making coffee, mixing cocktails, arranging flowers, or watering plants, the model can understand task goals, decompose them into multiple steps, and continuously observe the environment, predict changes, and adjust subsequent actions during execution.

Zhu Jun stated that Shengshu’s entry into world models was not about chasing industry trends, but a natural extension of the video foundation model technical pathway. It was precisely the long-accumulated video model capabilities that enabled the team to validate the path from video prediction to action generation earlier.

From Vidar to Motus to Motubrain, Shengshu has progressively extended video models from generating digital content to understanding and acting upon the physical world.

A core challenge facing embodied AI today is the high cost of collecting high-quality robot data, which is also easily constrained by robot embodiment, task type, and environment.

Traditional VLA models typically rely on teleoperation to collect robot trajectories, then use imitation learning to fit specific actions. This approach can achieve high precision in relatively constrained scenarios, but struggles to cover the variety of real-world conditions. When objects, backgrounds, or robot embodiments change, generalization often degrades significantly.

Shengshu has chosen a different path: first learning general world knowledge from massive, diverse, real video data, then using robot data to align specific skills and embodiments.

Zhu Jun describes this data system as a “data pyramid,” covering internet-scale general video, first-person human action video, simulation-synthesized data, and real robot trajectory data.

General video helps the model broadly learn about people, objects, environments, and patterns of motion; first-person video provides information closer to human manipulation; simulation and robot trajectory data further supplement embodiment state, control commands, and precise action feedback.

This training approach mirrors human learning. Before mastering a specific skill, a person has already built a foundational understanding of the world through long-term observation. When actually learning to ride a bike or use tools, only a small amount of specialized guidance is needed for rapid transfer.

Therefore, Shengshu pursues general knowledge and general skill learning first, followed by specialized skill training. With a strong general foundation, the model can reduce dependence on large-scale real robot data when adapting to new embodiments and tasks.

But having massive data is only the first step. More importantly, can the model truly make use of that data?

In the past, internet video was difficult for traditional VLA models to use directly because it lacked robot action annotations and control signals. Shengshu chose a generative technical pathway, using video prediction and reconstruction as learning objectives, allowing the model to learn about people, objects, space, actions, and state evolution from pixel-level changes.

In Zhu Jun’s view, the era of large models calls for more general, fundamental learning objectives. Language models learn linguistic patterns by predicting the next token; for video and world models, the most fundamental objective is predicting and reconstructing pixels.

Although pixel-level prediction is more difficult, it forces the model to simultaneously handle objects, space, time, actions, and state changes. The more complete and data-aligned the learning objective, the more likely it is to drive deeper world understanding.

As early as 2022, the team proposed and open-sourced the U-ViT architecture, introducing Transformer attention mechanisms into diffusion models, improving scalability and validating its Scaling Law. This laid the technical foundation for Shengshu’s subsequent large-scale video and world model training.

Data determines how much world knowledge a model can learn; architecture determines whether that knowledge can be unified and deployed.

Currently, many embodied AI solutions still use relatively separated structures: a vision model for understanding the environment, a world model for predicting the future, and a policy model for outputting actions.

This modular approach is easier to implement for specific tasks, but independently trained modules tend to cause information loss and error accumulation, and struggle to form a general foundation that can scale and transfer across tasks.

To address this, Shengshu independently proposed and continues to evolve Mixture-of-Transformer (MoT) — a unified architecture.

In this architecture, understanding experts perceive and interpret the environment, generation experts predict future states, and action experts generate robot actions. Different experts collaborate through joint attention mechanisms, achieving integrated understanding, prediction, and action within a single model.

Built on the same foundation, the model can generate and predict video in pixel space, output executable actions in action space, and simultaneously predict future states and actions.

Shengshu is not simply attaching an action module to a video model, but aims to bridge the digital and physical worlds from the architectural level — through unified architecture, pixel-level learning objectives, and end-to-end global optimization — building a scalable general foundation.

Motubrain is built on the MoT unified architecture. In a single model, it unifies environmental understanding, future state prediction, and action trajectory generation.

Facing complex long-horizon tasks like flower arranging or watering, Motubrain first understands the current environment, task goals, and object relationships, then plans the task, predicts environmental changes from different actions, and generates executable robot actions.

During execution, the model does not output a fixed trajectory. Instead, it continuously observes the environment, predicts the future, and adjusts actions based on actual states. In other words, as the robot acts, the model simultaneously “imagines” possible outcomes, forming a closed loop of understanding, prediction, and action.

More importantly, Motubrain explores not training separate specialized models for individual tasks and embodiments, but achieving capability reuse on a unified foundation, enabling world knowledge and action capabilities to transfer across tasks, environments, and robot embodiments.

In over 50 complex tasks in RoboTwin 2.0’s randomized environments, Motubrain achieved an average success rate exceeding 95%, demonstrating the potential of the unified world model approach for multi-task generalization.

For Shengshu, Vidu, Vidu S1, and Motubrain are not separate products, but manifestations of general world model capabilities at different stages and in different scenarios.

Vidu generates dynamic content in pixel space, learning how people, objects, and environments change over time; Vidu S1 further introduces real-time input, enabling the model to continuously understand instructions, update states, and respond during generation; Motubrain translates the model’s understanding of the environment and predictions of future states into executable robot actions.

The three correspond to the progressive evolution of world model capabilities: generating the world, interacting with the world in real time, and ultimately acting upon the world.

In Zhu Jun’s view, future, more powerful world model foundations are expected to simultaneously support a wide range of intelligent tasks in both the digital and physical worlds. As foundation model capabilities continue to improve, agents may gradually move from passively responding to instructions toward autonomously discovering, planning, and executing tasks.

General intelligence in the physical world will not arrive suddenly at a single moment, but will consist of a series of milestone stages: from completing diverse tasks in relatively constrained spaces, to achieving stable generalization in more open environments, to autonomously discovering and completing tasks based on environment and goals.

From video generation to real-time interaction to acting in the physical world, Shengshu is advancing along a clear technical pathway, driving models from generating content toward understanding the world.

Looking ahead, Shengshu will continue to focus on general world models, improving model generalization across different tasks, environments, and robot embodiments through large-scale pretraining and efficient post-training — enabling AI not only to generate the world, but also to understand it and act within it.

CCTV live replay: https://w.yangshipin.cn/video?type=2&vid=2031507201&pid=600225972