In October 2024, Machine Heart reported on a key model work by the Qing team: RDT-1B.
At the time, while still a PhD student in China, Ph.D. candidate Kim Jong-il proposed RDT-1B, released a month later by π0, to combine large-scale multi-robot data pre-training with Diffusion Transformer, to pioneer the advancement of the basic model of arm robot scattering to one billion parameters.
The RDT-1B demonstrates the Zero-shot generalization of unseen objects and scenes, as well as the Few-shot learning ability of new skills, and is a representative work of the physically intelligent exploration of large-scale pre-training paradigms.
Two years later, the Physical AI company LiberAI, founded by Phnom Penh, released a new generation of world action model Liber-0.
It's not just the model architecture that has changed: from millions of robot demonstrations to 2.2 million hours of human experience; from claw-and-claws to high-freedom hands; to the generalization of never-before-seen objects and scenes, to the zero-shot execution of complex multi-step tasks.
This time around, LiberAI has also released a very rich technical report with experimental details.
In addition to Zero-shot real-machine testing, the report systematically examined the scaling efficiency of different data sources, the ability of humans to demonstrate the migration of new skills to robots, and the impact of physical shape diversity on model performance.
It tries to answer not just how many tasks robots can accomplish today, but a more fundamental question: Can massive human experience, along a verifiable, scalable Scaling path, continue to transform into the real capability of robots to operate?
01
GPT randomly assigns 60 questions
Zero-shot begins to transcend long-range maneuvers
A direct way to test whether a robot is capable of generalization is to get it to perform new tasks that it has not been specially trained in.
LiberAI this time assigned the title to GPT-6.
The team produced 60 GPT-6 tasks, divided into three levels based on operational difficulty, with 20 items per group.
The first group is Pick-and-Place (PnP), the ability to grab, move and place the test base.
The second group is Non-PnP, which involves more complex object interactions, including pressing, pulling, joining, and folding.
The third group is further upgraded to a challenging multi-step, long-range Non-PnP task, where robots need to continuously coordinate multiple subtle actions to achieve their ultimate goals.
The key limitation of the test is that there is no robot demonstration training for these targeted tasks.
For each mission, 10 machine experiments were conducted, totaling 600 times.
Liber-0 ultimately achieved a combined success rate of 63.5%, 34.0% and 10.5%, with an overall success rate of 36.0%.[3]
The most worthy of attention is the third group.
For robots, multi-step tasks are far more than an increase in the number of movements. Each touch changes the state of the object, and the first move also creates conditions for subsequent operations.
For example, in food packaging tasks, robots need to open soft paper bags first, hold the bags together with their hands, and then put items in different shapes one by one, before finally sorting and folding the bags.
As the paper bags deform with contact, robots must adjust their movements in continuous operation, rather than simply performing a fixed path.
While the success rate of 10.5% is still in its early stages, this has already been demonstrated: the model begins to be able to fully execute some of the challenging long-range agile tasks in the absence of a robot demonstration of targeted tasks.
Zero-shot generalization is extending from a single action to more complex operations.
The team also compared Liber-0 and GPT-6 Astra directly as a robot strategy.
In the same group of missions, the success rates of the three Astra-only groups were 8.3%, 5.0% and 1.7% respectively.
Experiments have shown that while general reasoning models are able to understand tasks and planning steps, they still face challenges such as exposure control, precision of action, and real-time feedback when performing precise physical operations directly.
Liber-0 has supported collaboration with higher-level agents such as Astra, where the Agent tracks mission status, adjusts intermediate targets, and produces specific actions by Liber-0.
When general planning capabilities are combined with generalizable physical execution capabilities, robots begin to become more specific in their technical paths of autonomously completing unknown complex tasks.
02
More than public test results
LiberAI also opens its Zero-shot simulator to everyone.
There is also a very interesting design for this release.
In addition to the Zero-shot evaluation of 60 virtual machines, LiberAI has also launched an online interactive simulator:
I'm going to tell you something.
Compared to model APIs, the use threshold for online simulators is lower.
By uploading images of objects to a web page, generating desktop scenes, selecting targets and moving locations, you can run model reasoning and observe Liber-0's performance in a simulated environment.
From custom-testing objects to observing a model complete a task, the entire process can be done directly in the browser.
This means that the test subjects of the model are no longer entirely pre-selected by the development team.
Users can upload objects themselves and test the model's adaptability to different shapes and combinations of objects.
Of course, this set of simulators is not the same as the previous 60 real-machine tests. According to the official report, the simulator uses models trained after 100 hours of remote data simulation, primarily exploring Zero-shot migration of objects rather than real-machine generalization of complex long-range tasks.
But the two are complementary.
On the one hand, the Benchmark is a precise task classification, with a quantified success rate; on the other hand, the user can customize objects, directly interacting with an open test environment.
LiberAI has also released a Zero-shot rating display page.
Not only did it showcase a few successful demos, but it gave test methods, quantified results, and open interactive inputs, giving the outside world the opportunity to look further at the real limits of the model's capabilities.
For a basic robot model, this verifiability itself is worth considering.
03
10,000 hours of glove data
Why is there more than 2 million hours of Internet video?
If the Zero-shot test answers what the hydrogen model can do, then a more valuable part of the Liber-0 technical report is the systematic study of where hydrogen capacity comes from.
In recent years, from the EgoScale to DYNA-2, more and more studies have begun using large-scale human first-person video training robots.
Human video is rich in objects, scenes and operational experience, and naturally has the potential to scale.
But LiberAI poses a more specific question:
How much data can contribute to the robot's ability to operate, and how much can be solved by increasing the size of the video?
The teams used Internet human video, open-source Ego data, vendor Ego data, cross-bodied robot data, and self-developed glove data for pre-training and post-training, respectively, using the same robot data, to compare their contributions to PnP and Non-PnP tasks.
The experiment used an average progress score of 0 to 100 points to measure the overall completion of the task.
The results appear on a very interesting Scaling curve.
In the comparison of data sources, glove data showed particularly significant performance improvements.
As the size of the glove data expanded from 1,000 hours to 10,000 hours, PnP and Non-PnP averaged improvements of 14.7 and 13.1 points, respectively.
What's even more surprising is:
Over 10,000 hours of handheld data, and more than 2 million hours of Internet human video on both Benchmarks.
The data differed by 200 times.
This is not to say that Internet video has no value. On the contrary, the complete Liber-0 model uses multiple data mixing training, with overall performance significantly superior to any single data source.
A really important finding is that while Internet video can provide a wide range of physical experiences, precise operation also requires another type of critical oversight: accurate motion information.
Human video can allow models to observe how a hand turns the bottle cap, but not necessarily to be able to accurately restore each finger's posture, movement trajectory and contact relationship.
It is easy to make mistakes with video reconstruction alone, especially when faced with shadows, changes in perspective and microscopic operations.
LiberAI's solution is the self-developed LiberSense Glove.
This flexible glove is only 0.3 millimeters thick and weighs 131 g. It uses first-person and dual-wrist cameras to record hand gestures and movement tracks to capture human-seen scenes and actions.
What's more, it doesn't need robots to pick it up.
People can wear gloves directly and carry out daily operations in the kitchen, office, store and industrial environment, leaving high-quality movement data.
These data provide reliable Action Grounding for mass human videos that lack precise action tags.
Internet video is responsible for expanding the coverage of physical experiences, while high-quality handheld data is responsible for making these experiences truly connected to executable actions.
LiberAI's experiments also confirmed a key finding: that human data with precise controlled movements can demonstrate much higher data efficiency than ordinary video in terms of sophistication.
04
Gloves not only improve performance
We can also teach robots new skills directly.
If the data scaling experiment explains why gloves are important, the next experiment addresses the question of whether human data can replace real-machine demonstrations.
Traditional robot learning relies heavily on remote data. The more complex the task, the greater the flexibility of the hands, and the higher the cost and difficulty of data collection.
LiberAI has designed a set of direct control experiments.
The researchers reduced the robot sample to 25% of the original data pool and added human action data to the glove collection.
The results showed that 25% of real-machine samples combined with complete glove data performed more than the baseline using 100% of real-machine samples.
In this group of tasks and training conditions, the glove data effectively reduced reliance on expensive real-machine data.
However, there is another group of experiments that is more proportionate.
The team further asks: If there is no skill at all in the robot demonstration, can human action data directly teach this skill to robots?
In the experiment, the robot demonstration only covered PnP and did not include operations of the Non-PnP type.
At this time, the model's average progress score on Non-PnP tasks was only 2.2 points.
After adding Non-PnP data collected from human gloves, the model's performance improved significantly; further adding to the complete glove data, the Non-PnP score reached 54.6, which is close to 56.3 for full robot demonstration training.
Robots do not accept real-world models of the corresponding categories, but are able to learn new sophisticated operating skills from human glove data.
It should be pointed out that these skills have never appeared in all training data, but come from demonstrations of human action rather than corresponding robot demonstrations.
This difference is the key to the experiment.
In the past, human video was often viewed as ancillary data for robot pre-training; while Liber-0 experiments show that high-quality human action data has begun to play a role in transmitting new skills directly to robots.
If this capability can continue to expand, with a large number of humans skilled in operations that robots have not yet learned, there will be an opportunity to model through human demonstration.
From relying on the accumulation of experience by robots themselves to acquiring new skills directly from human experience, the scaling-up of space for robotic learning has also been further opened up.
05
From physical form to model Recipe
How is LiberAI's full-fledged Scaling Engine built?
High-quality motion data alone is not enough to complete the migration from humans to robots.
One fundamental problem is that humans and robots have different bodies.
Different people's palms and fingers vary in length and movement, and the geometry of the robot is not the same as the range of joint activity.
Traditional methods often require re-mapping human actions to robots.
LiberAI attempts to make models learn these differences directly.
In Liber-0, the team introduced the Unified Embodiment representation, using human geometry, hand structure, and robotic motion information as context input models.
Models not only learn what movements should be performed, but also learn how different bodies should perform the same tasks.
The team also designed a set of Morphology Scaling experiments: keeping the total amount of glove data almost unchanged, only increasing the number of participants in the sample.
As a result, under the conditions of handheld data training alone, the sample increased from 1 to 100 and the average robot score increased from 12.7 to 40.1.
Even without increasing the total amount of data, it can only enhance the diversity of body shapes and lead to significant performance gains.
This provides an experimental basis for Embodiment as a new Scaling dimension.
Liber-0 uses the World Action Model architecture and uses three expert streams of language, vision, action and Multimodal Attention to jointly predict future images and action.
Pre-training is divided into three phases: first through extensive human video learning visual prediction, followed by combined learning of vision and movement, and finally further enhancing the training weight of glove and remote control data, enhancing fine-grained operation.
At the end of the third stage, the model did not require additional post-training, with PnP and Non-PnP averaging 81.4 and 72.3; continued post-training only improved by 2.2 and 1.5 points respectively.
This indicates that within the scope of the reporting tests, a substantial part of the robot's operational capabilities have already been developed in the pre-training phase.
From data quality, action monitoring, physical diversity, to model architecture, to training Recipe and self-development, LiberAI is integrating key components that affect human experience migration efficiency into a complete suite of Scaling Engines.
More importantly, these designs are not limited to technical claims but are supported by contrasting experiments respectively.
This is the most valuable part of this Liber-0 technical report: not only publishing the effects of the model, but also the source of the ability to explain it as much as possible, and what factors drive the model's continued growth.
06
From RDT-1B to Liber-0:
How far have we come in two years?
If you go back in time to 2024, and look at today's Liber-0, you will see some interesting changes.
Two years ago, RDT-1B attempted to provide better generalization capabilities to two-armed robots through large-scale robotic data pre-training with Diffusion Transformer.
A year later, RDT2 began to shift its attention further to non-core data. The team improved UMI's data collection equipment, accumulating more than 10,000 hours of human operational data, and exploring cross-core Zero-shot deployments that do not rely on targeted robot demonstrations.
Liber-0 is another step forward in the technological path.
Data sources come from a combination of human experience shifting to the Internet scale and high-quality handheld action supervision; the development of operating terminals from clutches to high-freedom hands; the evolution of model architectures from Diffusion Transformers to the World Action Model driven by Mixture-of-Transformers; and the broadening of robots' objectives, from the completion of relatively short-range operations, to further extending to the sampling of complex long-range tasks.
From RDT-1B to RDT2, and then to Liber-0, the technological evolution trajectory, which in a way is also a summary of the development of physical intelligence over the past two years, is always clear:
How can robots face unseen objects, unknown tasks and open environments with stronger instructions to follow, zero sample generalization, agility and long-range execution capabilities?
Looking back at the exciting demo two years ago, hand-held, water-dropped and object-operated, it's no longer the end of the industry.
Today, large-scale pre-training has begun driving high-frequency maneuvers; robots have begun to be able to continuously complete more complex multi-step tasks; lightweight human embryos, scalable human data collection equipment, and a complete engineering pipeline from collection to training is also gradually maturing.
These are not only the capabilities of Liber-0 to demonstrate, but also the results of the joint advancement of the whole body-smart industry over the past two years.
But going far doesn't mean going far enough.
Robots still face a large number of unresolved problems when performing complex long-range tasks steadily in open environments, especially in the face of flexible objects, precise touch, touch and gravitational control.
Even though Liber-0 has achieved real success in challenging long-range Zero-shot missions, the 10.5% success rate is a reminder that the general agility of deployments that are reliable and scalable is not small.
This path, which can be difficult to follow, relies only on a stronger general reasoning model to directly solve all the problems of fine-grained action control in the real world.
Perhaps that's why this systematic disclosure of Scaling experiments by LiberAI is so important.
It attempts to turn some long-standing judgments that remain in the technical intuition into research questions that can be measured, controlled and continuously verified.
After a small amount of training, a mirror finished producing videos of different orders of milk tea, which made the distance between these studies and future applications more intuitive.
In retrospect, the paths that were once tiny, have finally coalesced into the ability to be seen, measured, and expanded today.
Every step of the way in the pursuit of universal physical intelligence, robust technological advances are bringing robots one step closer from understanding the world to truly participating in the world.
The link is as follows:
Liber-0 is the official technology blog:
Liber-0 Zero-shot rating:
The Liber-0 interaction simulator:
LiberAI's official website:
RDT-1B paper:
The main page of RDT2 project:https://rdt-robotics.github.io/rdt2/
© THE
