TRIAL 032× SPEED
Qwen (Fine-tuned)
FailedAdapter insertion
2× speed · muted
00:00HD ↗
Beyond vision / Grounded in touch
Touch-R1
Reinforcing Touch Reasoning in MLLMs
Turning contact evidence into physical understanding—across tactile perception, comparison, and reasoning.
TouchReason-1M / TouchReason-Bench
Research team & affiliations
1 Xiamen University 2 Great Bay University 3 Fudan University 4 Nanjing University 5 Daimon Robotics
THE GEOMETRY OF CONTACT FIG. 01
From contact to understanding.
Explore the object atlas ↗CONCEPT VISUALIZATION
NOT SENSOR MEASUREMENTS
Explore the object atlas ↗CONCEPT VISUALIZATION
NOT SENSOR MEASUREMENTS
PHYSICAL EVIDENCE. STRUCTURED REASONING.Discover the research
4optical tactile sensors
1M+synchronized tactile pairs
4,800held-out benchmark QA pairs
60.1TouchReason-Bench average
01 / The idea
Seeing is not
the whole story.
Touch-R1 brings R1-style reinforcement learning to tactile reasoning. We build TouchReason-1M, a multi-sensor dataset with synchronized tactile-force records and verified reasoning QA, and TouchReason-Bench, a benchmark for tactile perception, ordinal comparison, cross-sensor consistency, and visual-tactile conflict. Touch-R1 combines ordinal-aware accuracy, cross-sensor physical consistency, structured-format control, and input-side tactile grounding, enabling MLLMs to revise visual priors using contact evidence.
02 / ROBOT EVIDENCE
From contact to control.
Explore real-robot interaction, then compare model rollouts task by task.
IN SIMULATION / CASE EXPLORER
See the task. Compare the rollout.
Choose a case. Compare the same task across three methods.
CHOOSE A CASE1 LIBERO SPATIAL / 9 METAWORLD
LIBERO Spatial · Case 43 · Recorded 1×
Bowl to plate
pick up the black bowl between the plate and the ramekin and place it on the plate
01 / 10
Preview speed · shared by all methods
Selected simulation cases · Outcomes per recording · Inference waits omitted · Clips hold their final frame.
Recording protocol & case metadata ↗
These 10 examples were selected for Touch-R1 completion and Original Qwen non-completion: one LIBERO Spatial case and nine MetaWorld cases. They are curated demonstrations, not an aggregate benchmark success-rate sample.
GPT-6 Astra follows the model name supplied with these recordings; the backend model identity has not been independently verified. It uses zero-shot direct control; Qwen and Touch-R1 use action training, with different training budgets.
Source timing is preserved in the files: LIBERO 1× and MetaWorld 0.5×. The default 1× preview plays that source timing; the optional 2× preview multiplies all three methods equally without editing the recordings. Model-inference waiting time is omitted. Outcomes and action-step counts follow the supplied case manifest; clip duration is not an inference-speed measurement.
Source case metadata ↗Media provenance ↗03 / The method
Three-stage tactile-grounded training.
Learn contact dynamics, connect touch to language, then reinforce reasoning grounded in physical evidence.
Tactile Dynamics
Pretrain a touch encoder by predicting future tactile tokens from contact histories.
QA SFT
Align touch, vision, and language to a structured reasoning interface.
Tactile-adapted GRPO
Reward answers that are correct, consistent, parsable, and tactile-dependent.
Ordinal-aware accuracy
Cross-sensor consistency
Structured format
Tactile grounding
J_Touch-R1(theta) = J_GRPO(theta; R) + gamma G_tg(theta),
R = 0.6 R_acc + 0.3 R_cs + 0.1 R_fmt
04 / TouchReason-1M
Multi-sensor data for tactile reasoning.
TouchReason-1M couples tactile frames, deformation fields, shear, depth, force, metadata, ordinal labels, and reasoning QA.
THE ACQUISITION DEVICE
Built around physical contact.
A handheld acquisition setup combines an optical tactile sensor, a six-axis force/torque sensor, and an integrated IMU. A RealSense D405 provides the external view.
View original device figure ↗
THE CONTACT COLLECTION / FIVE PAIRED RECORDINGS
One contact.
Two perspectives.
Explore the external RGB view alongside the optical tactile response. Choose a sample below, then play both views together.
Sample 01
RGB / OPTICAL TACTILERGB view
Tactile view
SELECT A CONTACT SAMPLE01 / 05
Source acquisition footage. Each sample pairs the external scene with its optical tactile recording, showing the contact and sensor response together.
THE OBJECT ATLAS / 80 COLLECTION SAMPLES
A world you can touch.
Everyday objects. Real surfaces.
A closer look at the diversity behind TouchReason-1M.
Selected photographs from the acquisition collection.80 SAMPLES / 1,000+ OBJECTS IN THE DATASET
1,000+ objectsEveryday objects spanning nine material categories.
14,700 sequencesValid tactile contact sequences collected under a unified interaction protocol.
1.323M pairsSynchronized tactile data pairs with deformation, shear, depth, and contact-force signals.
4 sensorsDM-Tac X · GelSight Mini · Xense · Tac3D.
05 / The evidence
Stronger tactile reasoning across benchmark families.
Touch-R1-7B outperforms frontier MLLMs, general VLMs, and tactile-specialist baselines on TouchReason-Bench.
TouchReason-Bench
Tactile perception, comparison, and consistency.
60.1AVG / 100
Qwen2.5-VL-7B
28.0
GPT-4o
35.4
Octopi-13B
41.7
SToLa
47.5
Touch-R1-7B
60.1
Average scores on a 0–100 scale. Higher is better.
| Model | OMAE ↓ | L2-EM | CSC | Avg |
|---|---|---|---|---|
| Gemini-2.5-Pro | 0.67 | 13.8 | 41.8 | 37.6 |
| SToLa | 0.45 | 22.1 | 51.9 | 47.5 |
| Touch-R1-7B | 0.24 | 39.5 | 71.3 | 60.1 |
OMAE: ordinal error (lower is better). CSC: cross-sensor consistency.
QUALITATIVE COMPARISONS
Touch reasoning. In action.
Five cases. Three models. Actual answers.
Strong response, small protrusion
01 / 05THE DIFFERENCE
Both baselines select large. Touch-R1 selects small, matching the source label.
Curated tactile QA · Verbatim answers · Fixed text replay, not inference latency. Source details ↗
06 / A closer look
What makes tactile reasoning work?
Model-scale comparisons and controlled ablations isolate the contributions of training data and reward design.
07 / Build on this work
BibTeX
touch-r1.bib
@misc{lai2026touchr1reinforcingtouchreasoning,
title = {{Touch-R1}: Reinforcing Touch Reasoning in {MLLMs}},
author = {Yingxin Lai and Yafei Zhou and Fucai Zhu and Siyu Zhu and Weihao Yuan},
year = {2026},
eprint = {2605.27154},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2605.27154}
}