Henry from the Monastery of Phineas
QbitAI is public.
After a year and a half, Ingrid's next-generation basic robot model is finally here!
Recently, at this year's CORL conference in Austin, Ingrid GEAR, led by Jujuco and Jim Fan, brought out a new generation of world action models.
GR00T 2。
In the one and a half minute official demo, the GR00T 2 can use scissors cutting labels, install graphic cards, rub matches, chop bananas for breakfast, rub coffee, close tables, sell popcorn.
From hand-to-hand handling to full-body movement, the model's mission scope covers almost every aspect of the robot's capabilities.
Furthermore, according to Zhuyu, the models, data, simulation environments and data collection hardware of the GR00T2 will also be continuously open source.
At present, 100,000 hours of human data have been opened for the first time.
It's not too much to ask, is it?
GR00T 2: generalization, flexibility and adaptability of the expanded base model of robots
As a world-action model, the GR00T2 can simultaneously predict future images and robot movements. In training, it teaches how the world of aluminum will change and how aluminum and aluminum robots should act together.
At the conference, Jim Fan positioned it as a response fast enough to run locally, as well as a model of physical skills that is easy to demonstrate through demonstration churches and easily driven by reasoning models such as System 1.
Put it back inside the GR00T 2, which is what Juju said on Twitter about Xadapt's cross-bodied adaptation.
Simply put, in the past, teaching a robot to complete a new task often required re-collecting a batch of robot sample data for its physical shape and control methods.
X-Adapt is trying to break that limitation.Skills demonstrated by humans or handheld devices can be transferred directly to robots that have not been trained in this task.
Specifically, Invisalign shows three paths.
1、From human demonstrations to robots.
The researchers wore a motion-capture device to demonstrate how to place, and then the two-fingered Dexter could complete the same task.
Throughout the entire process, no remote robot samples were collected for the task. With human samples alone, Dexter had a success rate of 65% in 20 trials.
2 - Moving from hand claw to mechanical arm.
The researchers used a handheld UMI claw to demonstrate how to assemble a toy unloading truck. After studying the model, the skill was transferred to the YAM mechanical arm.
Despite the fact that UMI is different from YAM's claw shapes, YAM is still capable of completing assembly tasks that take about four minutes from end to end, with a success rate of 60%.
Similar migration paths are also used to convey tasks such as belt sorting and polishing, and dismantling of control panels.
3 - Moving from wearable skeletons to robotic hands.
Researchers wore the exoskeleton of the DexUMI hand to demonstrate the production of champagne yogurt and the harvesting of corn.
The model uses these data to fine-tune the robot to complete breakfast tasks without the need for additional robot remote-operation demonstrations to collect the task.
In summary, the three paths point to the same change:The same skill does not have to be demonstrated over and over again on every robot.
Of course, cross-body migration doesn't mean there's no need for robot data at all.
In human motion capture and UMI pathways, target robots will still provide data unrelated to specific tasks, allowing models to become familiar with their own physical structure, movement patterns, and control space. Demonstrations of new tasks can be derived from more easily collected human operations.
This is also the direction highlighted by the GR00T2 Technical Report.Expanding the generalization, flexibility and adaptability of the foundation model of robots.
In addition to migration, the GR00T 2 also demonstrates the ability to improve execution accuracy through a small number of robot demonstrations.
In another group of Dexter-50 assessments, the model's zero-sample average success rate was 40 percent; the success rate increased to 71 percent after one demonstration per task; and 95 percent after five demonstrations.
More importantly, the five demonstrations were less than three minutes in total. Subsequently, only about 30 minutes of LoRA mitigation was required on a single RTX 5090 to produce a model that matched the task.
In terms of actual calls, Invidia is also keeping pace with the trend, opening the WAM API.
Through APIs, rational models can be responsible for dismantling tasks, checking execution progress, and assigning specific physical skills to GR00T2 execution.
In other words, the GR00T2 can not only complete its skills on its own, but can also serve as a physical execution module for higher intelligence calls.
So, the question is: Invidia, how do you feel about the NBA?
How can 400,000 hours of human video be turned into the power of robots?
After the X-Adapt, there is first a data pyramid. GR00T2's pre-training combines three types of data:
402,730 hours of human first-person video;
10,184 hours of UMI presentation, covering 2,897 tasks;
23,500 hours of remote data by robots.
It's easy to see that human video is much larger than the other two categories. This means that robotic learning materials are no longer limited to the trajectory of man-made remotely collected action, but are further extended to a large number of real human activities:
How people pick up tools, move animals, complete assembly, and how the surrounding landscape changes after a single move.
While UMI and wearable devices are responsible for adding more direct operational trajectories, remote-operating robot data helps models learn how to control and control different body responses.
The team also conducted a series of large-scale experiments to verify whether human video could help robots work.
They used data training models that contained no human video, 1% and 10% and all human video, respectively, and evaluated them on 72 Dexter missions.
The results showed that with the increase in human video data, the average mission progress on real robots increased from 50% to 70%.
This group of experiments gave a fairly straightforward answer:Large-scale human experience, which not only enriched the visual knowledge of models, but also transformed the progress of robots in the execution of real tasks.
In order to consume the data, the GR00T2 receives language instructions, multi-point camera images, and robot status, and uses the same distributed Transformer to predict future video and action sequences.
Finally, around the GR00T2, Invidia is also taking a complete toolchain of data collection, processing, training, simulation verification and actual deployment.
It's all about collecting.They allow researchers to directly record human operations, especially those delicate movements that are difficult for traditional remote operations to do naturally.
It's a data center.Including Human-1M and Flora. The former provides human first-person videos, the first 100,000-hour sub-series has been released; the latter supports data compilation, descriptive markup, and automated evaluation.
It's a simulation.There is SimFoundry, which can reconstruct the task environment from real-world scenarios for strategic testing, data generation and skill development.
The execution end,It is made up of different models and components.
ASPIRE combines robot models with coding skills to support task execution and error recovery. Turbo uses lighter networks and more compact video representations to reduce predictive costs.
On the G1's Jetson Thor, Turbo's single forecast time is 124 milliseconds; SONIC runs on another Jetson Orin, with full-body control at 50Hz.
The former generates movement plans, while the latter transforms the plans into continuous physical movements that allow robots to maintain coordination while stretching their hands, bending their backs, and moving around.
With the GR00T2, developers can not only call the model itself, but also collect their own data along the supporting toolchain, adapt their own tasks, validate their skills in simulation, and finally deploy them to real robots.
This is also where Invisalign's announcement is more noteworthy:It opens up not just a robot model, but a whole development system from data collection, model training, to skill calling and actual execution.
One more thing
In a live talk at CoRL, Jim Fan said:
Astra didn't break our dreams, it broke our walls.
Wow, this isn't a cold winter, this is a robot summer.
While that's fine, today's release of Claude Fable 5.2 also highlights the performance of RoboDojo as a lesson in robotics.
In the RoboDojo Sim simulation two-arms test published by Anthropic, using the same operating framework, each model has the highest possible reasoning intensity, with Fable 5.2 scoring 64.9%, and GPT-6 Astra 29.8%.
If not, the next "Friendship Period", the Big Model Team and the Body Model Team will also be named.
Reference links
[1]
[2]
