Recently, Professor Tao Tao, the chief scientist of robotics, released a two-brain collaborative exploration: make one robot's exploration a basis for more robots; make today's experience a basis for reducing the mistakes that will have to be made tomorrow.
It's the first time I've ever been in a relationship with a woman, and it's the second time I've been in a relationship with a woman.
"by the last beginning where those before them had left off"
It's the first time I've seen this.
How can one robot make the same mistakes as another?
Imagine an ordinary cup-taking task: a robot finds a cup of water, plans in a good direction, but when it is lifted, it slips. It adjusts the position of contact, crawls again, and finally completes the task. For the user, this is the end of the story; for a system that can continue to learn, the question that is worth asking is just beginning: Why would it slip? Under what conditions is this adjustment effective?
A failure if only logs are left, and a success if only labels are left, similar trial errors may occur repeatedly as deployments expand, in different devices, and in different projects. Effective knowledge of interactions along with evidence and applicable conditions can be refined, stored, and validated in new tasks, so that real experience can gradually accumulate into a capacity.
This is also a growing concern with the development of physical intelligence today. General models are enhancing the ability of robots to understand goals, organize skills and handle complex tasks; world models provide new ways to predict the consequences of actions, compare candidate programs; perception and control technologies allow these capabilities to enter into real physical processes. As they are connected, we have the opportunity to ask further questions: can robots improve follow-up operations while completing tasks; can the cost of acquiring a new capability decrease as experience accumulates?
Real interaction and simulation play a complementary role here. The real world can expose conditions that models do not yet understand and provide a basis for physical judgment; calibrated simulation can extend exploration, help system comparisons, and spend limited real-time on problems that most need to be confirmed. The value of this cycle ultimately returns to real-world testing: whether predictions are more credible, whether failures are easier to recover, and whether new scenarios require less repetitive trial and error.
For China's physically intelligent industry, a more complete manufacturing and supply chain system, rich operating scenarios, provide important opportunities for this path. From assembly on production lines, to transportation in warehouses, to crawling under complex conditions, a large number of interactions are likely to reveal replicable physical relationships.
We want to connect SenseNova's mission understanding, reasoning, planning and capability organization, physical state judgement and action outcome prediction with the Enlightened World Model, and then, through the body Harness organization of perception, memory, skills and control, bring real feedback into the next judgement and into subsequent improvements in capabilities.
We hope that bi-brain collaboration will drive this accumulation: allowing one robot to explore and provide a basis for more robots; allowing the experience gained today to reduce the mistakes that must be repeated tomorrow.
One, from completing tasks to accumulating skills.
To make a successful grabbing experience replicable, the system needs further analysis: whether the contact position is appropriate, whether the grip is stable, what problems are solved by the movement adjustment; whether these judgments are still valid when objects or claws change.
At the same time, the sliding can be caused by different reasons - the state of the surface of the cup, the change in the load brought on by the liquid in the cup, the position of the contact of the claw, even the initial positioning error can affect the grappling results. A single observation is often insufficient to distinguish between these reasons. The system needs to link the original predictions, the actual actions performed and the feedback obtained, by supplementing the observation, the available contact information or targeted tests, to gradually reduce uncertainty. Lack of evidence should also be preserved as a validated hypothesis.
This step determines the form in which the experience can be retained. If subsequent verification finds that a short-distance test before lifting an object under certain objects and contact conditions helps to determine whether the grip is stable, then what should be retained include: what conditions the test applies to, what signals need to be observed, what actions should be adjusted when the result occurs, and what situations cannot still be judged accordingly.
A truly transferable experience is knowledge that is retained along with evidence and scope of application. It can be expressed as a recoverable skill, a recoverable experience record, or it can be trained into a strategy or a world model. Regardless of the form it takes, connections need to be established from interactive recording to subsequent behavioural improvement. The system should also identify which experiences provide new information: repeated completion of already mastered movements, with an interaction that reveals judgement defects, may have a very different learning value.
When another robot is faced with a similar task, past experience can help it find a solution that is worth trying more quickly, and may also suggest which conditions need to be prioritized. For example, awareness of grip stability may provide a reference for new tasks, specific grip strength, range of motion and control parameters need to be combined with new claw, sensors and execution capabilities. The meaning of experience replication is to reduce the scope of exploration, reduce repeat test errors, while maintaining the ability to judge new conditions.
This accumulation will ultimately need to be reflected in subsequent tasks. Under comparable conditions, learning a related skill requires less demonstration and real-machine experimentation; replacing an object or an entity, a faster adaptation process; confronting disturbances, finding biases and recovering more easily; and whether skills are still reliable after new capabilities are added. These results can help us determine whether the system really benefits from past interactions.
From task completion to accumulation, it means that every verified knowledge that a robot acquires has the opportunity to be the starting point for further learning. As these starting points abound, the system should be able to organize skills more effectively and explore unknown conditions more targetedly. To establish such a path, it is necessary to connect task understanding, physical judgment, action execution and experience learning.
Secondly, why do we use double-brain synthesis?
In order for past experience to really help the next action, robots need to answer two interrelated questions at once: What should the next step be in order to accomplish the goal? What could the next step lead to under the physical conditions at hand? The first concerns the goal, planning and skill organization, the second concerns the state, contact and consequences of action.
In well-known dual-brain collaboration design, the cloud brain SenseNova focuses on understanding tasks, organizing capabilities. It needs to judge what users want to accomplish, break down goals into executable steps, select the right approaches from existing experience and skills, and adjust plans when situations change. Faced with a failure, it also needs to combine the task process to complicate: whether the original goal is reasonable, whether skill selection is appropriate, which judgments need to be reaffirmed.
The Enlightenment World Model focuses on the understanding, generation and prediction of physical states. It provides a predictive basis for the possible consequences of different actions, based on the current history of observation and interaction, determining the location, movement and state of contact of objects, and, when necessary, constructing a candidate evolutionary process. These predictions should also retain uncertainty: which conditions have been fully observed, which judgments rely on unverified assumptions, and which places need to be further confirmed through new interactions.
This division is related to different requirements for resources and time. Task reasoning requires a broader pool of knowledge, memory, and skills that the cloud favors sharing and updating these capabilities; physical state judgement requires tight on-the-spot changes, and end-to-end deployments help to make timely use of new observations and feedback. Faster contact adjustment, gestural stability, and action execution are undertaken by corresponding strategies with real-time controllers. Model strategies and controllers each take on appropriate responsibilities to co-create a complete action process.
To take a cup task as an example, synergy can be developed as follows: SenseNova's brain chooses to take a cup skill based on the goal, and proposes approaching directions and grappling schemes; enlightenment combined with current observations, predicts the possible state changes that these schemes may bring. If the shadows make it difficult to judge the location of the contact, or if there is insufficient information to estimate the stability of the grappling, this uncertainty should be entered into the task decision. SenseNova thus chooses to change the perspective of observation, arrange a test, or adjust the grappling method.
The key to bi-brain collaboration is that task reasoning continues to receive physical feedback and that physical predictions are developed around the task. Candidate programs should integrate task logic with current objects, scenarios, and physical conditions; and physical predictions should explain how the outcomes affect the next choice. The two can use their respective capabilities to make the same decisions through clear interfacing of objectives, candidate actions, predictions, and uncertainties.
This collaboration should continue until the end of the task. Judgments made prior to execution, predictions made, actual actions taken and final results can form the basis of the complex. The discrepancies between prediction and reality can help the system identify the issues that need to be further investigated: whether the state is misunderstood, physical conditions have changed, skill selection is incorrect, or execution has not met the expectations. These clues are analyzed and verified, then decided to enter experience memory, skill adjustment or model training to provide a more reliable starting point for subsequent actions.
We focused on organizing these two types of capabilities with a pair of brains, while maintaining the space for implementation to evolve. Both can share representation and partial model capabilities, and specific model organizations work with the cloud and end-to-end, which should also be adjusted according to task, computational and time-delay requirements. Continuous testing is needed to see if this collaboration helps robots detect critical uncertainties earlier, choose more effective actions, and bring effective knowledge of real interactions to follow-up tasks.
In order for this information to be transmitted accurately between models, and to further connect perceptual tools, experience memory, skills calling and action execution, a complete systematic mechanism is needed.
Three, from model collaboration to system collaboration.
When the brain SenseNova proposes a grabbing scheme, after the Enlightenment gives a corresponding prediction, the system still needs to do a series of specific tasks: verify whether the current observations are enough to support the action, convert the targets into instructions that the robot can execute, call on the appropriate skills, and execute the deviation from the expected timely processing. Judgments between models need to be made through these links in order to become effective actions in the real world.
In well-known system design, the body Harness is responsible for organizing and coordinating tasks. It maintains task status, connects memory, skill libraries, perception, and 3D reconstruction tools, to clarify what information each module exchanges, under what conditions calls are made, and how to process the execution of feedback. Through this operating mechanism, the brain SenseNova's task planning, enlightened physical prediction and actual execution of the controller can correspond to the same task phase.
To take a cup, for example, the position of objects provided by the perception tool needs to be converted into the current robot's coordinates and action interface; the selected skills need to specify the applicable claws, input parameters and execution conditions; the system also needs to judge which observational representatives are crawling, which signals mean slipping or touching abnormalities. A skill that can be called steadily should be combined with these conditions and judgments into the skills library. Thus, SenseNova can know when a skill is available, where it is executed, and what recovery paths remain after failure.
Action execution also requires proper allocation. Learning strategies can combine vision, task objectives and historical status to generate action references, while parsing controllers are based on intrinsic characteristics, complete position tracking, contact adjustment, and execute constraint processing. Faster feedback is responded in the corresponding control loop; as the deviation affects the task, Harness transmits state changes back to SenseNova and Enlightenment, by which they re-judge whether to supplement, observe adjustment programs, or call recovery skills.
Harness also needs to retain a traceable basis for the accumulation of experience. Scenario conditions in a task, body configuration, skills used, model predictions, feedback and artificial interventions can all affect the interpretation of results. This information can be matched to specific execution phases to help identify problems from inadequate observation, predictive bias, skill selection, or execution, and thus select appropriate improvement objects.
After a successful on-site recovery, the result is an analytical experience. Whether it is useful for other tasks requires further verification. Therefore, the updating of corrective bias and follow-up capabilities in operation can be carried out at different rates: the current system completes the necessary adjustments first, the interaction record then enters screening and replication; the supported knowledge enters the memory, skills or model and can also modify its scope when evidence changes. The experience thus has a clear path from generation, verification to reuse.
When the robot's body changes, this mechanism is equally important. Common task descriptions and skill intentions need to be mapped through interfaces to new modes of perception, operating space and control capabilities; conditions related to contact, load and execution accuracy need to be recalibrated. Adaptation and replication should help the system to clarify which knowledge can continue to be used and which parts need to complement learning.
The value of system collaboration is to enable models, tools and controllers to evolve in their own right and to allow verified experiences to have continuous repetitive pathways. When perceptual tools are upgraded, strategy updates or robot interfaces change, the system can examine the affected skills and experiences, retain established knowledge, and validate parts of the changes.
Such system design provides the foundation for the deployment of bi-brain collaboration: connecting task planning, action execution, feedback logging and experience management, creating conditions for verification of follow-up capabilities and continuous improvement. On this basis, correction of current tasks, accumulation of long-term capabilities, and improvement of learning and R&D methods provide the opportunity to form interconnected, gradually deepened closures.
Four, three inbound and outbound circles:
Let action, learning and R&D promote each other
After real feedback enters the system, it can lead to different levels of change: how to correct prior actions, improve follow-up capabilities, and further optimize access capabilities.
The first closed loop is self-executing, allowing the robot to perform its current tasks more reliably.
The robot continuously receives new observations in action, according to which it updates status judgments, adjusts plans, and when necessary adds perception or calls recovery skills. When there is a shading, contact changes or slippage in the process of taking cups, the system needs to identify deviations in time, combining the current conditions to decide how to proceed. Faster execution feedback is processed by the corresponding control loop, and changes that affect the progress of the task enter SenseNova's synchronous judgment with Enlightenment.
This closed loop can run without changing the model parameters. It improves the decision-making and execution of the current task, allowing robots to take advantage of the information processing changes that have just been obtained. Task completion rate, deviation detection and recovery time, and amount of artificial intervention can all help to measure its effectiveness. The execution process leaves observations, predictions, movements, and results behind while providing a basis for further learning.
The second closed loop is experiential learning, allowing one interaction to improve the ability to follow-up.
After entering this layer, the system needs to filter out learning content from the execution records, analyze which new conditions they reveal, correct which old judgments, and then choose the right way to preserve those insights. Some experiences can first become long-term memories with the appropriate conditions, and are retrieved and summoned in subsequent tasks; some experiences need to be transformed into skill adjustments; and problems involving motion generation or physical prediction may require targeted training in strategies or world models.
For example, once the recovery method has been further verified, it may be effective for objects that are close to a type of contact condition. The system can preserve the relevant conditions, judge the basis and the recovery method, and prioritize references when encountering similar situations. It still needs to confirm the scope of these experiences and continue to modify them through new feedback.
Whether the learning loop is established to see whether the benefits can be extended to subsequent tasks. Whether the system reduces repetitive errors in independent testing, whether the adaptation is completed with fewer new examples and real-machine trials, and whether it is able to be maintained while acquiring new skills. These results determine whether an update is worth deploying and how much of this experience should be replicated.
The third closed loop is to return to self-improvement, so that methods of acquiring abilities can also be continuously optimized.
When the system is able to continue to execute and learn, there are even deeper questions: which failures are worth investigating the most, what data should be collected next, whether current training methods are effective, the same real time, how to bring more capacity gains. These questions refer to data selection, experiment design, training processes, evaluation mechanisms, and the way models and Harness are organized.
We want a gradual involvement of R&D entities in this work. It can summarize repeated failures, provide verifiable reasons for assumptions, select experiments that can differentiate between these assumptions, and propose candidate proposals for data replenishment, model training, or system adjustment. World models and calibrated simulations can help compare solutions, while real tasks are used to verify whether improvements can be made. Effective methods have been verified, and then move on to the next round of learning and R&D.
This layer requires more rigorous controls: whether, under comparable tasks and resource conditions, adjusted processes gain reliability faster, reduce manpower, computation or equipment time, and whether gains can continue to occur in subsequent tasks. Automation experiments can improve process efficiency; recurrent self-improvement also requires testing: whether systems can optimize experimental selection and learning methods based on results, and allow improvement capabilities to continue to participate in the next round of improvement.
This is the direction of research we want to continue to advance. It requires cross-task, cross-sectional evidence to gradually answer a more challenging question: can the system improve its own acquisition capacity while growing?
The three locks can run at different speeds. Correction biases in execution require timely response, long-term capability updates require verification, and changes in R&D methods require more complete comparison with replication. They share evidence of real interaction and also provide the conditions for each other: more reliable execution allows systems to access more tasks, more efficient learning reduces repetitive trials and errors, and better R&D methods improve the efficiency of the next round of improvement.
More valuable progress is to make it easier for the next step to happen. Bilateral collaboration hopes to create a pathway that extends from action to learning and extension to R&D. Whether this pathway continues depends on how we use limited real interactions and how we make imitating more targeted help for these interactions.
Five real machines and simulations:
Spending real time on critical issues
In real robots, there are three closing cycles that need to be faced with specific time and resource constraints. A single experiment may involve device modeling, scenario setting, artificial repositioning and failure recovery, and continued operation can also lead to wear and environmental changes. Therefore, improving learning efficiency requires careful selection of what questions to answer each interaction, and allowing the already acquired records to generate more training and verification value.
Real interaction provides the basis for the system's understanding of the physical world. For cup-taking tasks, three-dimensional reconstruction can help determine the shape, location and surrounding space of the cup; the conditions under which the effects of load, surface state, contact transformation, etc. can be captured, and it is also necessary to combine action with feedback to estimate. The system should gradually identify which parameters are closely related to the current task, which judgments have been supported, and which still have greater uncertainty.
These insights can be the starting point for constructing a simulation environment. How world models learn from interactive data about how the state changes, 3D reconstruction tools provide geometric information about scenes and objects, and physical simulators provide interveneable, repeatable experimental environments. They need to calibrate each other through real-world records: can prediction or simulation match key changes in real-world observation under close initial conditions and movements?
A real trajectory provides an opportunity to be the starting point for a set of controllable experiments. The system can change the position of objects, the range of loads, the contact conditions or perform disruptions, comparing which programs are still effective and which conditions will cause the original skills to fail. Simulation should also cover easy-to-mistake movements, help the system learn how to identify failures, when to recover, and under what conditions observations should be supplemented.
Simulation can also help select the next real-machine experiment. If several physical models that are compatible with existing observations give different predictions for the same action, this discrepancy points to a question worth investigating. The system can design new observations or experiments around it, prioritizing the conditions that are most likely to change the choice of action. Thus, real-machine interaction assumes both task execution and the role of eliminating critical uncertainty.
The Real2Sim2Real cycle that we hope to form follows this path: real interaction to provide evidence, model and simulation to establish and modify physical knowledge; simulation to expand practice, compare candidate programs, and expose the issues that need further confirmation; and screened programs to return to real-world testing. The new results continue to be used to calibrate environments, adjust strategies, and update experiences to build the next round of exploration on a more thorough understanding.
The effectiveness of real-world task testing improves the scope of simulation compared to practice. Therefore, evaluating a world model or simulation environment should consider its help in real-world action: whether it can correctly distinguish the advantages of candidate programs, whether it can predict critical contact failures, whether it can make real-world improvements in simulation, and whether it can reduce the real-world interaction tasks required to achieve equivalent capabilities.
Acceptance also requires the maintenance of independent real tasks and conditions to test whether the system can bring improvements to new objects, scenarios and entities. Total costs should also be calculated simultaneously, including new data, artificial operations, computation, equipment occupancy and recovery time. Only by linking these inputs to actual gains can one judge whether a simulation expansion or model update improves the efficiency of capacity acquisition.
As the cycle continues to progress, calibrated scenarios, verified physical conditions and reliable skill tests can gradually become the basis for subsequent collaborative tasks. Experience has helped the system to find directions worth trying more quickly, simulating the development of these directions in a more complete comparison, and real interaction to confirm which knowledge can be formed.
Six: Moving to the smart-body Transformer, the moment is coming.
The two-brain collaboration discussed above, the three progressive closing rings, and the mutual calibration of real and simulated machines ultimately point to a common goal: to allow robots to gain experience in real interaction as a basis for more tasks and a replicable body.
An important inspiration for Transformer is that the shared infrastructure can support multiple capabilities and sustain continuous expansion. We look forward to this transformational moment in time: the robotics space will gradually form a widely replicable common ground, allowing different tasks and entities to share verified physical knowledge, allow for more follow-up scenarios of accumulated experiences once deployed, and support models, tools, and controllers to evolve on this basis.
This objective faces more complex conditions in the physical world. Robots have different bodies, sensors and execution capabilities, and the same tasks are affected by changes in space, contact, load and environment. Therefore, a common underlying needs to connect shared physical relationships to their respective execution conditions.
We see the world model as the central link in building this underpinning. How objects move, how they touch, how they change their state, and what the consequences of an action can be, and these insights can provide a common basis for planning, controlling, imitating, and learning. If real interaction can continually calibrate these insights, then calibration models can help more tasks to select actions, organize practices, and discover unknown conditions, and physical experiences have the opportunity to enter a continuously expanding learning path.
Bi-brain collaboration is the systematic exploration of knowledge around the selection of this goal. SenseNova organizes tasks, skills and knowledge already available, provides insights and judgments about physical states and consequences of action, and physically Harness connects them to processes of perception, memory and execution. We hope that through such collaboration, feedback from a task can be analyzed, valuable insights verified, and accessed in appropriate form to subsequent tasks and model versions.
Whether this pathway can form a common foundation requires answers to several key questions. As experience covers more objects, scenarios, and contact conditions, can the same core mechanisms continue to benefit from more extensive independent tasks? Whether knowledge has been gained when switching to new ones to reduce the data and engineering inputs needed to adapt? Whether proven skills and experience can continue to play a role after updates to models and tools? These results determine whether the accumulation of systems can span single tasks, single devices, and single versions.
Particularly noteworthy is the change in the ability to access costs. Under comparable conditions, whether the actual machine time, manpower input and computational resources required to master a relevant new ability decrease as experience accumulates; whether the system can maintain the skills it already possesses and operate reliably under new conditions when it obtains these gains. Such evidence can show that a common foundation is generating sustained value if it is able to reappear across tasks, across entities, and across cycles.
This value will further change the way robotics develop. New entities can use existing task organizations and physical knowledge to address differences in their perception and execution in a more focused way; new applications can use verified skills, scenarios and assessment tools to find key issues to explore more quickly; and different teams can contribute to the results of clear interfaces and verification standards to allow their progress to be pursued more closely.
To form such a common foundation requires architectural exploration, but also scalable learning methods, reliable engineering mechanisms, and ongoing authentication. Whether bi-brain collaboration can contribute to this will ultimately be answered by these results: more and more robots will be able to enter new tasks with their experience, and will also be able to bring new discoveries back to a common foundation, so that subsequent capabilities have a better starting point.
Let's keep the next robot out of the way
Back to the original glass of water.
Once the task has been completed, the value of this interaction still has an opportunity to extend. One shift may reveal contact conditions that are not yet understood, and one adjustment may provide a method of recovery that is worth checking out. After analysis and verification, these experiences can help the system understand more clearly: under what conditions it can continue to act, when it needs to change strategy, and what questions must be answered by the next interaction.
Tomorrow, when another robot faces a similar task, it will need to observe its present environment, confirm its own conditions of execution, and can use its experience to find solutions worth trying, notice changes that are easily overlooked earlier, and organize trials more targetedly. When these insights help it reduce repetitive errors and shorten the adaptation process, today the interaction begins to generate value beyond a single task.
It is this continuous connection that bi-brain collaboration hopes to establish. Mission reasoning and physical prediction correct each other through real feedback, and the knowledge gained in execution is validated into further learning, learning and R&D methods are also improved in continuous practice. Different models, tools and robots themselves can evolve along this path and continue to use the already accumulated effective experience.
This expectation needs to be supported gradually by repeatable results: whether new tasks involve fewer real-machine experiments, whether new entities are adapted at a lower cost, whether systems remain reliable while growing, and whether these gains can be sustained as experience accumulates. To turn these issues into clear experiments, traceable evidence, and continuous improvement is well known to advance the work that bi-brain collaboration needs to do.
We hope that one day, when robots enter a new scene, they will be able to start with a fuller awareness; when they leave the scene, they will also be able to leave a test-driven experience for later generations.
Let the exploration of a robot save the cost of repeated trials and errors for more robots. This is the progress we hope that bi-brain collaboration will drive.
Author's note
Professor Tao Tao is the chief scientist of the Changzhou robotics, as well as a distinguished professor at the Australian Academy of Science and the South China University of Technology in Singapore. He is responsible for the research and development strategy of the Changzhou robotics, leading the team to realize its long-term vision of physical intelligence.
He is a world-renowned scholar in the field of artificial intelligence. He is the author of the most cited papers in the field of engineering since 2014 and the most cited papers in computer science since 2015; in 2019, he was selected by Stanford University as one of the top 2% of scientists in the world; in 2025, he was ranked number one in Singapore in the list of best computer scientists published by the internationally renowned academic platform Research.com.
He has published more than 300 papers in leading international journals and conferences and has been cited more than 170,000 times on Google Scholar, h-index = 197. He is a two-time winner of the Eureka Award in Australia (2015-2020), the President's Medal of the University of Technology Sydney (2015), the IEEE ICDM Research Contribution Award in 2018, and the IEEE CS Society Edward J McCluskey Technology Achievement Award in 2021.
Professor Tao received his bachelor's degree from the University of Science and Technology of China, his master's degree from the Chinese University of Hong Kong, and his doctorate from the University of London in the United Kingdom. After graduating, he taught at universities such as the Hong Kong University of Technology, the South China University of Technology, the University of Science and Technology of Sydney, and the University of Sydney.