Once again, the GM model has shown amazing potential in the field of robotics. Today, Anthropic's newest flagship Fable 5.2 comes out!
In the official presentation, the robot controlled by the model can easily complete a series of operational tasks such as solving puzzles, bottle caps, and painting robots.
In addition, Fable 5.2 scored 64.9% on the public RoboDojo benchmark test. According to Anthropic, this performance not only outperforms other cutting-edge models using the same testing framework, but also outperforms most AI systems developed specifically for robot control.
From the two more GPT-6 Astra models to the current Claude Fable 5.2, they showcased a flowery operational capability that allowed the industry to see more potential for generic large-scale model-driven robots.
But does the success of the demonstration and the high scores in the simulation test mean that the robot has mastered a truly universal ability to operate physically?
When a robot "reaches out" and faces a task that requires precise operation, testing has just begun.
For example, in the case of bottle caps, the robot's fingers must accurately locate the grip position, apply the appropriate force, and adjust the movements in a timely manner according to how much the bottle cap rotates, how much the fingertips slide. Several millimeters of displacement, and subtle changes in force, determine whether an operation is successful.
To accomplish these precise operations, robots need to work together in a variety of physical models, including visual, spatial, and tactile. For this, an important technological route for physical intelligence is to start from the interaction between the robot body and real physics, building a model that is native to the physical world.
At home, the penguin wave is moving in this direction.
The team continues to build a system of native models around the physical perception, understanding and operation of robots. The touch base model LingBot-Touch, released today, is a concrete practice in the field of high-precision agility operations.
In the process, the team collected nearly 5,000 hours of real-world touch data, covering three different sensing technologies for magnetic touch, compressive resistance and visual touch, allowing the model to learn the common laws of touch behind different sensors. These touch characteristics can also be accessed by different robotic motion models and replicated in different agile hands and real-world operations.
The main page of the project:
Technical report:
It's a great way to get started.
Hugging Face：
What is the meaning of this word?
For this purpose, LingBot-Touch builds a complete set of technologies around the pre-training of asymmetrical touch data, unified image learning and downstream motion model access.
Following are a few key links.
5000 hours of real data
Let's share the "hands" of different robots
Touch seems like just perceptual touch, and to actually make it a general foundation model is far more complicated than imagined.
The need for touch varies from robot to robot when performing different operations. Fingers pinch a glass ball beam to accurately capture the tiny touch positions and force changes; hands hold larger objects to master the power distribution in different regions. Different operating locations, different tasks, sensing accuracy, range of coverage and volume require different sensors.
Currently, there are several technical routes for touch sensors, and LingBot-Touch focuses on three categories: visual touch, magnetic touch, and pressure-resistant touch.
Through internal cameras, visual touch can observe changes in the shape of elastic surfaces, capturing details such as contact positions and contours; magnetic touch senses the contact and force of magnetic field changes caused by soft-mass magnetic material changes; and compressive touch measures the pressure distribution of different locations by measuring changes in resistance to material pressures.
The three types of sensors are perceived as physical changes that occur when the fingers are in contact with objects, but the underlying measurement principles and signal expression are very different. There are also differences in the number of sensing units, spatial layout and sampling frequency of different devices.
This poses a difficult question to the foundation model of touch: how do we get the data collected by different sensors to participate in training together?
If special tactile models are trained for each sensor, it is difficult to transfer the experience accumulated by one robot directly to another. Even if the operating model already has a strong operating capability, it may be necessary to reconfigure the handling of the tactile signal due to the replacement of the sensor.
The team is trying to figure out how to transform the tactile experience accumulated by different hardware, different tasks, into basic capabilities that can be replicated.
To do this, the team designed a multi-branch coding, shared mainstream architecture and developed the Tactile-JEPA training method for tactile signals using the JEPA (Joint-Embedding Predictive Architecture) approach.
The first problem to be solved is the problem of spatial alignment between different sensors.
The number, arrangement and spatial resolution of different Taxel touch sensors are different. If these data are directly entered into the same Transformer, it is difficult for the model to know exactly where one of the sensing units on the different sensors corresponds to the finger.
The team created a unified set of standardized point geometry coordinates to map the sensing units of different sensors into a common three-dimensional space.
On this basis, each Touch Token has a five-dimensional address (time window index 1D, finger number 1D, three-dimensional spatial coordinate 3D), consisting of time window index, finger number and three-dimensional spatial coordinate.
In coding mode, LingBot-Touch does not require all three sensors to use the same primitive signal processing method, but instead uses a lightweight Tokenizer designed for pressure-resistant, magnetic touch and visual touch, which converts their primitive signal into a touch Token under a unified interface.
Subsequently, these tokens entered the shared Haptic Transformer (HaT), which learns the contact laws behind different sensor signals from the same mainstream network.
This design retains the measurement characteristics of different sensors while enabling them to participate in model training together. Differences between sensors are mainly handled by their respective coding branches, while higher-level touch characteristics are learned by the shared mainstream.
The question of how to align the data is addressed, and then the answer is: how exactly should models learn touch?
LingBot-Touch uses Tactile-JEPA to predict characteristics in a potential space. Simply put, the model first obscures the partial touch Token, processing the remaining visible information by the Context Encoder; at the same time, the Target Encoder processes the complete touch sequence, generating target characteristics.
Among them, the parameters of the Target Encoder are updated by moving the EMA average of the Index of Context Encoder, not directly involved in gradient reverse propagation. The model simultaneously monitors the prediction results of the position being obscured and the position being visible.
This design is related to the characteristics of the touch data itself.
Touch data is very rare, most of the time the fingers are not in contact with objects, and there is a large amount of near-zero background signals in the sensor record. If the original signal is directly optimized, the model may focus a great deal of effort on predicting these background data.
Potential spatial prediction means that models don't have to spend a lot of energy to recover uncontacted background signals, but instead represent characteristics associated with learning and contact positions, power sizes and their dynamic changes through time-space correlation in real interaction data.
To train the model, the team also built a multi-source touch data set that collected 4,983 hours of real touch data, covering 464,431 interaction segments, about 1.3 billion tons.
The data comes from compressive, magnetic and visual touch sensors, data covering grip, rotation, sliding, wiping, assembly, hand-to-hand coordination, and manipulation of fragile, deformable objects, and includes two sources:
One is that human operators who wear touch gloves generate the main part of the data set.
The other part comes from remotely operated robots that use Sharpa Wave's sophisticated two-armed system.
Data collected by touch gloves account for two-thirds of the total and data collected by remote operations for one-third.
Here's a thought worth considering: the training of tactile base models doesn't have to rely entirely on the robot itself to collect data.
People who wear touch gloves can gain a lot of exposure experience in their daily operations. Furthermore, real interactive information on specific hardware is supplemented by remote-controlled robot data, and data from different sources have the opportunity to participate in foundation model training together.
This provides a new pathway to touch data scaling.
26 real-world missions
Verify whether the general touch can actually migrate
The team evaluated LingBot-Touch's downstream capabilities from three levels: tactile understanding, robot operations in real environments, and robot operations in simulated environments. The experiment covered two handsets, Sharpa Wave and XHand1, which contained 26 real-machine operations and 12 simulation operations.
Let's see if the model really understands touch.
First, the team evaluated LingBot-Touch's freezing encoder in four dimensions (contact size, spatial geometry structure, time dynamics, and intermodal correspondence) and designed a total of five tasks.
The results showed that using only 1000 hours of pre-training data, the LingBot-Touch was able to demonstrate stronger performance in five missions. At 30 to 50 N, the model predicted a contact force error of less than 2 N; in spatial perception, the contact center positioning error was sub-millimetre, and the contact area estimation error was less than 2%; in time prediction, the future size of contact force was also predicted to be less than 2 N.
The team further evaluated the quality of LingBot-Touch's features through visualization.
During the contact phase, different characteristic points form a smooth, elongated structure that corresponds to the gradual change of the tactile signal. As the tactile response of different fingers increases, decreases, and the force is redistributed between different fingers, the corresponding contact phase is also distributed along these structures in adjacent areas.
This phenomenon suggests that the LingBot-Touch is not only able to distinguish between contact and non-contact states, but also retains information that changes continuously during contact, capturing the process of tactile response in an interaction that evolves over time.
How do robots behave from touch to real action?
The team then applied LingBot-Touch to real-world robot strategies to test whether this knowledge could be transformed into practical operational capabilities.
The real robot experiment covers Sharpa Wave and XHand1. The team chose Diffusion Policy,π0.5,LingBot-VLA and T-Rex, and trained the LingBot-Touch as an additional touch representation module to access the downstream.
The two skilled hands completed a total of 26 machine-operation missions involving the rotation of objects, precise grabbing, manipulation of fragile and deformable objects, and scenes of arm synchronization.
Experimental results show that the operating success rates of different action models have improved significantly after accessing LingBot-Touch.
In Sharpa Wave, the average success rate of π0.5 increased from 47.56% to 68.58%, LingBot-VLA increased from 45.60% to 72.51%, and T-Rex increased from 44.06% to 70.05%.
On XHand1, the average success rate ofπ0.5 increased from 49.71% to 74.17%, and LingBot-VLA increased from 57.21% to 75.04%.[3]
Furthermore, this improvement is not limited to one model of action. Whether Diffusion Policy, π.5, or LingBot-VLA, T-Rex, the same set of pre-trained touch characteristics can be accessed by different architectural strategic models to provide touch sensing capabilities without having to re-train a set of touch encoders for each model of action.
Can tactile expression play a role in moving from the real world to simulation?
The team further expanded the testing environment to simulate, examining whether pre-trained tactile knowledge can be transferred to different physical environments, and whether reinforcement learning strategies can really help learn precise operations.
As shown in Table 2, LingBot-Touch achieved the highest success rate and cumulative returns in all 12 missions on both Sharpa Wave and XHand1 platforms. It achieved an average success rate of 68.95%, higher than CNN's 45.59% for touch feature programs and 21.42% for non-touch input programs, up 23.36 and 47.53 percent, respectively.
The advantage of LingBot-Touch is particularly evident in tasks that require synchronous control based on constantly changing contact states. On the Sharpa Wave platform, the model achieved a success rate of 97.4% in the disc rotation task, 97.1% in the pen rotation task, and 14.5% and 32.9 percentage points higher than the best baseline solutions in the corresponding task, respectively.
The spherical balance mission further reflects the importance of high-quality tactile representation. LingBot-Touch achieved 67.7% and 35.9% success rates on Sharpa Wave and XHand1, respectively, while the best baseline schemes on the two platforms were 32.4% and 10.0%, respectively.
These results suggest that the introduction of tactile observation to robots alone does not guarantee improved operational efficiency and reliability. The quality of the tactile expression itself is key to improving performance.
The training curve below also shows that in the Sharpa Wave rotation mission, the two sleek ball balancing missions, and the XHand1 blind grabbing missions, LingBot-Touch allows reinforcement learning strategies to achieve higher cumulative returns early in training.
This training trend further illustrates that the tactile features provided by LingBot-Touch reduce the optimization burden of learning tactics on how to use tactile information and help improve the stability of tactical training.
Where is the advantage compared to the already tactile model?
The team also compared LingBot-Touch directly to FTP-1.
In five touch-understanding missions, LingBot-Touch performed better on both Sharpa Wave and XHand1 platforms.
For example, the average error on Sharpa Wave decreased from 2.31 N on FTP-1 to 0.81 N and on XHand1 from 3.72 N to 1.32 N.
The gap in spatial perception is more pronounced. On Sharpa Wave, the contact center positioning error decreased from 2.19 mm to 0.12 mm; on XHand1 it decreased from 5.89 mm to 0.16 mm.
In touch video retrieval tasks, LingBot-Touch's Top-5 Recall improved by 56.13 and 41.13 percentage points, respectively, compared to FTP-1.
The team also conducted post-training on five real robot-operating missions, using two touch encoders to access the LingBot-VLA respectively. Finally, the LingBot-Touch increased the average success rate from 55.9% of the original strategy to 82.2%, and the FTP-1 corresponding score to 67.5%.
The more data, the stronger the model will continue to be.
Next, the team examined LingBot-Touch's scaling capabilities.
The experiment set up the LingBot-Touch pre-training dataset in five different sizes, increasing gradually from 1000 hours to 5000 hours, while keeping the proportion of data sources unchanged.
In the downstream tactile understanding mission, with pre-training data increasing from 1,000 hours to 4,000 hours, HaT-Tiny continues to improve its understanding of the four aspects of spatial structure, time dynamics, contact size, and intermodal relationships.
When the data scale increases to 5,000 hours, the model's performance is essentially evenly matched with the results below the 4,000-hour scale. This suggests that at this data scale, model capacity may begin to be a limiting factor for further performance improvement.
The team subsequently expanded the model's backbone network and upgraded the model to HaT-Small. The experiment found that larger models achieved further improvements under different data scales, particularly in continuing to use increased pre-training data to improve performance.
Real-world robotics also show a similar trend in scale expansion.
It's a real thing.
Looking for Shared Physical Laws
Every time the contact state, power, or slippage changes, the robot has to capture this physical information in time to re-judge the next move.
Seems to be a simple operation that requires multiple capabilities to work together. Space perception models judge the position and distance of objects, touch models capture contact with force changes, world models are responsible for learning how the environment changes with time and movement, and motion models generate and adjust movements based on this information.
This is the overall design idea of the brain-generalizing Brainstorming robot. In its technological system, space and touch perception, world modeling and motion generation are provided by their respective models, and through integration and collaboration between models, support the entire chain from perception, prediction to execution.
This architecture also reflects the technological idea behind the creation of native general-purpose body-to-body intelligence:
Although the layout, structure, and control of the sensors are different, the physical laws facing different robots have something in common. The foundation model needs to refine the laws that can be used by different hardware while preserving the necessary adaptation space for the specific body.
LingBot-VLA has validated the feasibility of "one-brain multi-machine", and LingBot-Touch pushes this idea to a more difficult unified touch model. With the flexibility to replace touch sensors, the perception model doesn't necessarily need to be trained from scratch.
Of course, this ability to replicate is not unconditional at the moment. True machine verification for the LingBot-Touch is mainly focused on Sharpa Wave and XHand, both of which are sophisticated, with pre-training not yet covering the capacitive sensors. More robotic shapes, complex environments and long-range tasks require more extensive experimentation.
This exploration also raises two key questions: which laws can be shared between different robots and which capabilities must be re-learned in conjunction with specific entities?
If the boundaries between the two are clear, robots will have the opportunity to reduce the cost of re-training basic perception models for different hardware, allowing more real interaction experiences to flow between different devices.
