Taken from the Temple of Fujian
QbitAI is public.
How is it that even AI has begun to grind itself, while robots are still learning crazy lessons about human movements?
Now in the AI circle, Agent, Coding Agent, RSI (recurring to self-improvement) these concepts are hotter than hotter.
A friend who is physically intelligent feels the pressure, even a hidden anxiety of "being overwhelmed by the general big model".
A few days ago, Claude also stated that Fable 5.2 demonstrates the ability to control physical robots without specific training for specific tasks.
Google DeepMind then released a new generation of VLA models, dubbed the most powerful to date.
Behind a fierce technological competition, the focus of manufacturers' competition has shifted to the bottom line.The data。
Currently, the industry-recognised efficient solution is to collect real human-operated data (Human Data) and use mass-reality interactions to generate general robots.
It's a good idea, let's see how easy it is.
The video captures the operation process, and when the fingers are obscured, the subtle movements may not be visible; even when the action is fully captured, it is difficult to judge when the contact occurs, how much force is used, based on the image alone.
If you put a handheld sensor on the device, it may affect the natural operation, material transformation, and signal interference can cause tactile data to be lost.
To put it another way, a task, from getting closer to the target, adjusting the body to completing the operation, has a whole set of movements that need to be recorded.
In response to a robbery in the Human Database, critical information was leaked, which led to the decision to manipulate the data.
On 9th November, at the 2026 CoRL (Robot Learning Conference)It's a smart robot.Two new products under DAS 2.0 have been released, with the intention of finding solutions from underlying hardware and systems.
One is a first-person point of view multi-mode data collection device.DAS Ego 2.0The other is a handheld data collection glove.DAS Dex 2.0。
In addition to the Ego+Wrist Camera (wrist camera) dual perspective collection program, which is currently the focus of many leading AI laboratories, the first perspective adds hand details that are easy to miss.
Here, human "small actions" that are easily overlooked have the potential to be studied by robots.
Human Data, too, has to evolve.
In other words, Human Data isn't just a camera, it's a process of recording human activity.
It's also a human operation, recording what information robots can finally learn, which may be a long way off.
And the data required for different tasks is very different.
Some need to see the entire operation process, some have to point fingers at the tiny movements, and others, just watching video is not enough to measure real contact.
But you can't always give every collector a piece of equipment to collect all the data, right?
The simplest way to do this is to make different collection devices grow and work together.
Some are responsible for the overall look, some specialize in the details of the palms of the hands, some measure real contact. These training signals can also be coordinated in time and space when collecting together.
Currently, its Gen DAS data collection system provides three main configurations.
Ego is responsible for the first-person perspective, recording the environment, actions, and mission processes of human operations, including different scenarios, operating habits, and even adjustments after operations fail.
If the camera on the head is too far away from the hand, a Wrist Camera can be added on the basis of the Ego, which is equivalent to an "exclusive position" for the hand to operate.
With the camera on the head, you can see people reaching out and picking up the parts, a handshake, but you don't have to see how the parts are oriented, and what fine adjustments have been made to the fingers.
The wrist camera is closer to the operating area and can continuously record the relative movement, approach direction, and local shape changes between the hands and objects seen in the picture.
Combined with camera marking and trajectory reconstruction, it can also match the movements recorded from different perspectives.
Going back and forth to the task that requires precise measurement of finger movements and real contact, you have to ask for the Dex data glove.
The combination of Ego+Dex can directly record joint movement and tactile information, adding to the testing of physical contact that is difficult to judge visually.
Three configurations, covering the mission context, local action and original biology measurements respectively.
In this system, the first upgrade is the "eye" of the robot.
The most intuitive change in DAS Ego 2.0 is that the camera can play better.
The system is equipped with six full-frame fast-door cameras, supporting 30fps synchronous imaging, and extending the horizontal field of view to 340°.
What is 340 degrees?
The horizontal direction is close to the circumference of the circle. With the exception of a small portion of the back, most of the surrounding directions can be covered by a lens.
While human activity may focus on the parts in front of it, the environment also influences the next step.
Taking items off the shelf, turning to another workstation, how to move along the way, whether there are any obstacles around, where the target is, may affect the success of the task.
With broader visibility, devices have the opportunity to retain more environmental information and action clues at the same time.
The robot learns later, and it doesn't have to be just those few seconds.
In addition to the view, another major upgrade is the camera pixel enhancement.
Of the six cameras, two of the main cameras are upgraded to 5 million pixels, while the other four are upgraded to 2 million pixels, with fingerprint tracking accuracy of up to 4mm.
In the same field of view and shooting distance, the sample density of the image width and high direction increased by about 58% from 2 to 5 million pixels.
These tiny structures on the fingers, edges of the objects, provide a wealth of image information.
However, model training is not about making high-definition documentaries. New details, if not used in model training, are also busy.
How to make these 5 million pixels work? There are really two different ways.
Article 1: Improving the label of supervision.
The picture is clearer, and the algorithm can more accurately locate key points in the hand, rebuild movement trajectories, and reduce deviations when marking movements.
Even the use of low-resolution images in model training can benefit from more reliable action labels.
The second is to make the model actually see more detail.
Otherwise, after working hard to make high-definition videos, the key changes between the fingers and the objects were compressed into small graphs during training.
In the model training phase, it can be done by local cuts, multi-dimensional inputs, and so on, as much as possible to retain critical visual information in the hand operating area, so that new pixels can play a role.
However, if you want to use these details in the training, you have to leave the picture in full.
Half of the equipment is out of power, or the hair is stuck with the lens, and the paint quality is still high and white.
Ego 2.0 supports continuous replacement of magnetic suction battery packs, with a single packet life of about 2.5 to 3 hours.
The supporting app is capable of checking head posture, recognizing hair covering, and has a keyboard recording, standard MCAP format, and after batch processing.
At first glance, it's almost as if it's not a particularly cool technique.
But it's true that when dozens, hundreds of people are collecting data at the same time, less interruption, less ineffective video, the collection process is more unified and the quality of data is more guaranteed.
These tiny improvements, accumulated, are real efficiencies and quality gaps.
Currently, the amount of "effective data" delivered by Ego 2.0 in a single quarter has increased from 100,000 hours to millions of hours.
How can real touch on the fingers be accurate?
Of course, the headset can play again, and there are some things you can't do yourself.
Especially the moment your finger touches the object.
Did you slip the bottle cap? Did you hold the glass tight enough when you picked it up? These changes are hidden on the touch surface, which the camera may not see.
It's been a long time since we've found that it's hard to get the real "hand sense" of humans back.
First of all, the device itself is not messed up.
If the conventional outer bone is too thick, it can easily occupy the operating space of the fingers and palms.
Human hands don't know how to handle nature, and the movements that robots learn, don't they taste from the source?
Dex 2.0 simply moves the main outer skeletal structure to the sides of the fingers, bringing the mechanical joints closer to the axis of movement of the human hand, giving space to the back, thumb and palm, reducing interference with natural operations.
Finger activity is normal, and sensors may also "distort the signal".
Once a finger is stretched out, the sensor can respond. Sometimes it only touches one location, and the adjacent touchpoint also responds.
That's right, guys, it's not touched yet, the sensors are excited, and that doesn't give the model a "touch illusion".
Dex 2.0 replaces the self-developed semi-flexible pressure-resistant electronic skin, which focuses on solving the problem of easily misrepresented sensors, trying to differentiate between the material shape, the signal generated by the channel interference, and physical contact.
Specifically, the hardware is equipped with 16 touch sensors for a total of 980 touch sensing units.
The sensing space resolution is 1mm, the minimum detection power is 0.1N, and the sampling frequency is 100Hz.
In addition, Dex 2.0 also features a self-developed "micro magnetic encoder", which is recorded by 21 single-handed independent joint encoders, supporting 21DoF motion capture, with a finger position accuracy rating of less than 1mm.
There are more detailed measurements of where the finger touched the object, how the contact strength changed.
Two big problems have been solved, and there is another connection, which is also testing equipment.
It's easy to get a few demos, but it's not the same for a month.
If you measure well today, the same movements tomorrow, the touch signals may change.
To make touch a real learning tool for robots, one must consider long-term stability in addition to one-time accuracy.
Dex 2.0 continued to concentrate for a month after its completion, and the tactile response to repeated actions was consistent under the same conditions.
Touch senses are measured and measured, and then we have to think about how to use that data on robots.
Dex 2.0 supports both in-person and remote-controlled collection modes.
These are real touch data, and Intel wants them to do something even more interesting: teach visual models.
Remember the previously mentioned Ego+Wrist Camera?
The head and wrist cameras record the entire task, while the wrist cameras move with their hands, observing the subtle changes between human hands and objects.
For example, pushing parts into a card slot.
The wrist lens can see how the parts are approaching the cartridge, whether there is relative displacement of the edges, or even some visible local shape change.
But how exactly do these changes in the images correspond to the state of contact?
It's not a good idea to judge by the camera.
Intel is currently exploring synchronous pairing of Ego+Wrist Camera+Dex, matching first-person perspective, wrist close-ups with connective tissue and tactile data from Dex detections.
This is equivalent to preparing a visual model for an answering exercise.
Through camera marking, trajectory reconstruction and multi-point alignment, the model has the opportunity to learn how the fingers actually make contact when changes occur in the image.
It also transfers information learned from real touch to lighter visual programs through a combination of supervision and distillation of knowledge.
If the route runs, even without the Dex glove, the Ego+Wrist Camera alone will provide an opportunity to estimate whether there is contact, contact area, and gain pixel-level precision hand tracking.
It is even possible to explore a step further: from judging "not touching now" to predicting "what happens next".
It's kind of funny, haha, that high-precision touch data has the opportunity to be "upgraded" along with low-cost visual data!
But at the moment, it is still an exploratory technological route.
Information such as object weight, hidden contact, etc. cannot be judged from images. Model predicted contact states cannot directly replace actual measurements.
Real touch can teach you how much "hand sensing" to see, and ultimately how the model is trained and performs the actual task.
From one hand to a whole bunch of moves
If the task on the fingers is done, can robots learn how to work with humans?
It's a long way off.
To carry the box, humans walk past, find a good stop, then bend down, reach out, grab the box, and finally get up and move away.
Just recording those seconds of hand operation, the robot may have learned how to grab, how to approach, how to get dry from behind, or a fog of water.
The Wholebody (full body action) was incorporated into the DAS data system in an attempt to replace these easy-to-miss processes.
It is divided into three levels:
- The strategy layer: recording how people approach the target, choosing a station, and when to adjust the observation position.
- Coordination layer: recording how the shoulders, arms and arms work together, completing one operation.
- The execution layer: records how objects rise, turn, move, and connect back and forth after being picked up.
In this way, robots learn more than just the movements of their hands.
But human movements are complete, and robots cannot directly copy.
Human robots, mobile mechanical arms, claws in fixed positions, physical structure and mobility are not small differences.
The movements that humans can easily complete, in exchange for different robots, must also be re-adjusted according to their respective movement constraints.
If you want to copy human work, you have to look at your "body plate" and not allow it.
This also means that the original data should be as complete and accurate as possible.
Time can't go wrong, especially with images, joint movements and touch signals from different devices.
Imagine that in the video, your finger touches an object and the touch signal is a bit late.
This is equivalent to the model getting the "standard answer" and the resulting answer is the wrong question.
Although DAS Ego 2.0 and Dex 2.0 are two separate devices, they can achieve a head-to-head time of about 1ms through a uniform clock, hardware trigger and camera exposure control.
How important is 1ms?
Calculated at 0.5 m/s of hand movement, a time lapse of 1 m corresponds to a distance of about 0.5 mm; a lapse of 10 m corresponds to about 5 mm.
This is merely a geometric bias caused by time misplacement, not an overall positioning error of the device.
However, a few milliseconds of difference between the touch, spin, and loose-hand movements can cause the motion in the image and the touch state detected by the sensor to be different.
If the model is more precise, the timing is inconsistent, and the model may also learn to run.
Currently, intelligent data products have been validated in models from teams such as Simate, Natural Wills, Dreamlabs, Biologics and Vidu AI.
Well, even the AI model R&D team has dropped the data, and the pressure is getting bigger and bigger.
But don't rush to judge who wins or loses.
From visual, action to real touch, Human Data is becoming more and more sophisticated.
Next, let's see who can really use this data to learn the truth about humanity.
The general model has gotten the topic out of the way, people of the machine, it's a turnaround.
