It looks like hard plastic, but is very soft to press. When vision and touch conflict, humans tend to rely on touch to correct visual distances; however, existing multimodal models often rely more heavily on vision, thus misjudging the physical properties of objects.
Figure 1 Research overview: Touch-R1 performs better on the TouchReason-Bench touch reasoning evaluation benchmark than other representative multilingual language models (left); Touch-R1 overturns visual misjudgements based on touch evidence, judging the material to be clay (right)
The DAI robotics team, together with Shenzhen University, Nanjing University, Greater Bay Area University, and Purdue University, has proposed Touch-R1, which will use reinforced learning for cross-sensory touch logic. The paper has been received by NeurIPS 2026. The team has also built the TouchReason-1M data set with the TouchReason-Bench benchmark, supporting training and evaluation.
The paper is titled: Touch-R1: Reinforcing Touch Reasoning in MLLMs
Address of the paper:https://arxiv.org/abs/2605.27154
The main page of the project:https://laiyingxin2.github.io/Touch-R1/
The code is:https://github.com/laiyingxin2/Touch-R1
The data collection:https://huggingface.co/datasets/laiyingxin/TouchReason
As the first tactile-determined physics R1 model, the study introduced for the first time reinforced learning in a systematic way to the tactile reasoning model, allowing the model to realize physical reasoning based on the tactile nature of the transceiver, transforming the physical cognitive capability of touch from what it looks like to what it actually looks like to what it actually is, providing a new path to building more reliable tactile reasoning.
The first tactile-focused physical reasoning model to build physical common sense for physical intelligence
Human judgments of the physical properties of objects, usually seen first and then confirmed or corrected by touch. This loop of perception, verification, and correction is an important way of interacting with the physical world.
However, today's large-scale models still tend to be more focused on vision. They are able to identify objects with difficulty in detecting hardness, roughness, friction, deformation, and force, and are more likely to fail in the face of transparency, reflection, or obscured objects. Tests have shown that when visual and tactile information conflict, models like GPT-4o, Gemini-2.5-Pro tend to be more confident in the shape of the lens than in the shape of the lens. Such misjudgments can lead to robots grabbing failures, damaging objects, and even risking safety.
To make the model read touch, there are three major difficulties: one is that touch data is scarce and touch physical reasoning data is less; second is that physical properties such as soft hard and rough are often sequential rather than categorical; and third is to understand the signals of different sensors and make judgments consistent.
The first tactile data system to create a complete infrastructure for tactile reasoning
The team first constructed TouchReason-1M, a large, cross-material, cross-sensor touch-reasoning data set, which was the first touch-data system for a physical model, combining multi-sensor visual touch data, sequential attribute tags, and verifiable reasoning queries. The data set covers more than 1,000 everyday objects and 9 major materials, bringing together the four visual touch-sensors DM-Tac XGel, Sight Mini, Xense, and Tac3D. Unlike previous touch-data sets, all objects are collected under strictly controlled standard operations and standard forces, allowing for horizontal comparisons of different objects.
TouchReason-1M focuses on visual touch perception, which provides high-resolution tactile data for large model tactile reasoning. However, the elasticity of different sensors, marking points, light paths, and structures vary, and the exposure to the same object will have different image performances. TouchReason-1M does not have a simple pairing of data, but a unified approach: from initial contact pressing to 20N, and then completing six exploration movements with standard force, and simultaneously recording original touch images, shape changes, cutting edge changes, depth changes, and other data.
Figure 3 TouchReason-1M data set for tactile reasoning
At the model level, the Touch-R1 is trained in three stages: first, by predicting future tactile vibrations, learning the dynamic laws of contact; then, by supervising structured questionnaires and tweaking, forming a perceptual - comparative - chain of reasoning that concludes with vibrations; and finally, using GRPO reinforcement with tactile design.
Touch-R1 further aligns sensor-to-sensor touch perception from the synthetic physical representation to synthetic physical properties logic: models analyze physical properties from the touch images of different sensors and make judgments, and bind the judgments by means of cross-sensor consistency rewards, thereby neutralizing differential sensors and obtaining true stable physical properties. At the same time, models also introduce sequential rewards to help models understand the degree of physical properties, such as hardness and roughness; structured format rewards and touch input interference to ensure models are reasoned based on touch only, reducing visual proofs to guess answers and answers, but also to understand touch shortcuts.
Figure 4 Touch-R1 Training process in three stages
Touch-R1's nine performances are significantly ahead of Touch-R1's performance.
To validate the model's capabilities, the team simultaneously constructed the TouchReason-Bench touch-reasoning assessment benchmark, identifying the physical properties, sequential reasoning, comparison reasoning, cross-sensor consistency, and visual-conflict-sensitive processing of the touch-reasoning model, and generalizing the object-reasoning model that has not been seen in 200 training stages.
Figure 5-TouchReason overview of the building process of the Q&A pair
On this benchmark, Touch-R1-7B scored best on all nine core indicators, including physical property recognition, sequence reasoning, and cross-sensor consistency, with an overall score of 60.1, 18.4 points higher than touch-specific model Octopi-13B, 24.7 points higher than GPT-4o, and 12.6 points higher than previously strongest touch-specific model SToLa.
Figure 6-Touch-R1-7B is leading globally in multiple core indicators
Touch-R1 also demonstrates how to touch-correct visual misjudgments: for objects that look plastic in appearance, the model is able to combine touch shapes and forces - depth curves, the objects' smoothness of recognition, and thus overturn visual precedents, correcting the material judgments to be tangible. This reflects the most important ability of Touch-R1 to not only receive vision and touch at the same time, but also to judge which information is more trustworthy when evidence conflicts.
Figure 7-Touch-R1-7B Questions and Answers
Meltdown experiments further identified the contributions of various parts of the Touch-R1 architecture. Cold-start monitoring, standard GRPO, sequential perception rewards, cross-sensor physical consistency constraints, structured format controls, etc. all brought varying degrees of improvement to the model's performance. At the same time, as the model grew in size, multiple indicators were steadily improved, showing the generalization potential of the Touch-R1.
The value of the Touch-R1 goes far beyond giving the model the ability to tactile reasoning. Through a complete infrastructure of data sets, models and evaluation benchmarks, the Touch-R1 promotes the transformation of touch from assisted perception to a core model of a body-based model. At the same time, the Touch-R1 is based on physical cognition consistent across sensors, laying the foundation for the cross-hardware migration of touch data and model capabilities, and provides the underlying capabilities for real physical interaction scenes such as material identification, quality testing, precision assembly, fragment tracking and flexible object operations.
Introduction by the author
The authors of the paper are from Xiamen University, Nanjing University, University of the Greater Bay Area, University of Transylvania and Daihatsu Robotics. The first author is Ray Manchu, currently a Ph.D. student at Xiamen University and a Daihatsu robotics intern; the first co-author is Zhou Xiafei, currently a M.S. student at the University of the Greater Bay Area and a Daihatsu robotics intern; and the author of the communication is Ms. Yang, currently an associate professor at Nanjing University and the chief AI scientist at Daihatsu Robotics.
