Picking up a raw egg takes precise force. The fingers need enough pressure to keep it from slipping, but not enough to crack it. Twisting a wet towel calls for constant adjustment as water leaves the fabric. Humans do this almost automatically. A camera can record hand motion, but it does not capture where the fingers make contact or how much force they apply.
Researchers from Texas A&M University, Google DeepMind, Carnegie Mellon University, Stanford University, the University of California, Berkeley, Yale University, Microsoft, NVIDIA, the University of Liverpool, Meta, the University of Washington, Northwestern University, Sony, Georgia Tech, and data supplier Overfit Lab have now released TouchScale, a 500-hour human vision-touch dataset built under a single sensor setup and a unified capture and synchronization process. The team set out to test whether scaling high-quality egocentric visual-tactile data to hundreds of hours would produce the same kind of gains seen in large video datasets, and whether those gains would carry over to robot manipulation.
What TouchScale includes
The paper highlights three main points.
- First, TouchScale contains 500 hours of synchronized human bimanual visual-tactile data, all collected with the same wearable system. It records head-mounted RGB-D, RGB video from both wrists, and full-hand tactile signals from both hands.
- Second, the work suggests that human touch can transfer to robots. Without human action labels and without mapping human hand motion to robot motion, mid-training on the human visual-tactile data raised the average success rate of four real-world robot contact tasks from 22.5% to 57.5%.
- Third, the dataset shows visible scaling gains. As more TouchScale data is used for mid-training, real-world manipulation improves. The same upward trend also appears in a separate zero-shot tactile prediction setting.
The paper is titled TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning.
Paper: https://arxiv.org/abs/2610.10288
Project page: https://touch-scale.github.io/
Dataset: https://huggingface.co/datasets/2077AIDataFoundation/TouchScale
The challenge was not just collecting more hours
According to the researchers, the hard part was not simply pushing the total to 500 hours. The key was extending the dataset while keeping the sensor hardware, synchronization method, and collection process fixed. Earlier public human visual-tactile datasets were mostly measured in a few hours to a few dozen hours, with many under 30 hours. If several small datasets are merged, hardware and collection procedures change alongside data volume, making it difficult to isolate the effect of scale itself.
TouchScale addresses that from the data collection stage. A head-mounted RGB-D camera records the overall task and scene depth. RGB cameras on both wrists capture close-up hand-object interactions. Both hands wear full-hand tactile gloves, and each glove has 880 tactile sensing units, or taxels, across the fingers and palm to record normal pressure during contact.
To maintain quality at scale, the team combines hardware constraints with algorithmic checks. Data collection uses high-precision, low-noise flexible skin tactile sensors to track pressure changes in the fingers and palms during interaction. After collection, a VLM-assisted automatic quality-control pipeline uses wrist video alone to judge grasping, releasing, and which fingers take part in contact, then cross-checks that against the tactile response. Samples with clear sensor failures are removed directly. Cases involving missing finger signals, severe occlusion, or weak visual evidence are routed to manual review.
What was captured in 500 hours
TouchScale includes about 87,000 interaction recordings, around 2,000 task descriptions, and more than 1,500 objects. It covers nine categories of collection environments, more than 800 scene configurations, and involves about 20 data collectors.
The tasks range from everyday actions to finer-grained contact behavior and tool use. The examples listed in the paper include pouring water from a kettle into a wide-mouth bottle, which requires continuous adjustment of container pose; rolling up a laptop charging cable and securing it with a hook-and-loop strap, which involves two-handed coordination and sustained contact; and sealing the top seam of a cardboard box with tape, which combines tool grasping, positioning, and continuous pressing. Different tasks, objects, and execution styles produce different hand-object poses and tactile contact patterns.
Compared with earlier public human visual-tactile datasets, TouchScale does more than expand the scale to 500 hours. It keeps the sensor hardware and collection process consistent throughout and provides bimanual full-hand tactile capture, with 880 taxels per hand. That makes it possible to study the effect of scale on its own.
Better cross-sensor generalization at the same 16-hour training size
The first experiment looks at zero-shot cross-sensor tactile prediction. A model predicts hand tactile signals from egocentric and wrist video, then is tested directly on EgoTactile, which was collected with a different tactile glove, without fine-tuning on the target data. Because the gloves use different sensor layouts, results are mapped to a common set of 12 anatomical hand regions for evaluation.
When training data is held to roughly 16 hours in each case, the model trained on a TouchScale subset reaches a contact cIoU of 0.181, above EgoTouch at 0.134. That comparison removes the simple "more data" explanation and indicates that TouchScale offers stronger cross-sensor generalization even at the same training size.
Tactile supervision also helps visual learning
The second experiment uses touch as a supervision signal for a visual encoder, then transfers the learned visual representation to action recognition. The researchers pretrain with TouchScale, OpenTouch, FEEL, and EgoTouch while keeping initialization and training budget unchanged.
Across MECCANO, Something-Something V2, and Ego-Exo4D, the visual encoder pretrained on TouchScale delivers the highest accuracy under both linear probing and end-to-end fine-tuning, outperforming pretraining with OpenTouch, FEEL, and EgoTouch.
Real-world robot tests: no human action labels, but large gains
The real-world experiments use an xArm6 robotic arm and a BrainCo Revo 2 dexterous hand. The setup includes four contact-intensive tasks: sorting soft and rigid objects, removing bottle caps, transferring test tubes, and wiping a whiteboard. Each task uses 50 robot demonstrations and 20 test runs.
The comparison is tightly controlled. Both versions start from the same N0-VTLA checkpoint. Both use visual and tactile inputs. Both use the same robot demonstrations for post-training. The only difference is that one version goes through TouchScale mid-training before post-training.
That mid-training stage does not use human action labels and does not map human hand trajectories to robot actions. It keeps the future tactile prediction objective from N0-VTLA, asking the model to predict later tactile changes from current visual, language, and tactile inputs. In the authors' framing, the robot is not learning how a human hand should move. It is learning how contact unfolds.
With TouchScale mid-training added, average success across the four tasks rises from 22.5% to 57.5%. All four tasks improve:
- Sorting soft and rigid objects: 10% to 60%
- Bottle cap removal: 40% to 70%
- Test tube transfer: 30% to 60%
- Whiteboard wiping: 10% to 40%
More data keeps improving both the model and the robot
The upward trend does not stop once the dataset reaches hundreds of hours.
In the tactile prediction task, the model and evaluation setup stay fixed while only the amount of TouchScale training data increases. As usage expands from 10% of the dataset to 100%, cIoU rises from 0.311 to 0.383. The result suggests that exposure to more human visual-tactile interactions leads to more accurate contact prediction, even on an unseen tactile sensor.
The same pattern shows up in robot control. The amount of real-world robot demonstration data stays unchanged. Only the amount of human data used in TouchScale mid-training increases, and the average success rate across the four tasks moves from 22.5% to 57.5%.
What the paper argues
TouchScale is built to answer a specific question: if sensor hardware and collection procedures stay fixed, what happens when the scale of human visual-tactile data grows on its own? Based on the results reported in the paper, larger data volume improves cross-sensor tactile prediction and leads to stronger real-world robot manipulation after TouchScale mid-training.
The authors argue that touch no longer has to remain an expensive and fragmented add-on modality in robotic systems. It can be collected at scale, learned at scale, and turned into physical interaction experience for robots, much like first-person video. In that sense, TouchScale is not only a larger dataset. It is also a data-scaling route in which the tactile feel of human operation is systematically recorded and then used to teach robots transferable interaction knowledge without action labels.
The paper also notes the current limits. The real-world evaluation covers one robot platform and four contact-heavy tasks. How human tactile experience turns into better robot control remains an open research question.
Authors
The paper lists Dayou Li, a PhD student at Texas A&M University, Hao Wang, a senior research engineer at Google DeepMind, Qianqian Yang, a PhD student at Carnegie Mellon University, and Zihao Zhu, a PhD student at Texas A&M University, as co-first authors. The corresponding author is Zhiwen Fan, an assistant professor at Texas A&M University whose research covers machine learning and embodied intelligence.
The original Chinese article states that the piece came from the WeChat public account Jiqizhixin.