TouchAnything: A Dataset and Framework for Bimanual Tactile Estimation from Egocentric Video

• URL: http://arxiv.org/abs/2605.13083v1
• Date: Wed, 13 May 2026 06:54:36 GMT
• Title: TouchAnything: A Dataset and Framework for Bimanual Tactile Estimation from Egocentric Video
• Abstract summary: EgoTouch is a large-scale egocentric dataset with dense tactile supervision for hand-object interaction.<n>We introduce TouchAnything, a vision-to-touch prediction framework that uses the egocentric view as the primary input.<n>We will publicly release the dataset, code, and benchmark.
• Score: 20.373348802426143
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Egocentric human video data, which captures rich human-environment interactions and can be collected at scale, has become a key driver of embodied intelligence research. However, existing egocentric datasets typically lack tactile sensing, a critical modality that provides direct cues about contact, force, and pressure in human-object interaction. Without such signals, models struggle to learn physically grounded representations of real-world interaction dynamics. While tactile sensors provide these cues, deploying high-quality tactile hardware at scale remains expensive and cumbersome. This raises a central question: can tactile feedback be inferred directly from visual observations, enabling scalable tactile supervision for egocentric video data and supporting physically grounded embodied learning? To enable research in this direction, we introduce EgoTouch, a large-scale multi-view egocentric dataset with dense tactile supervision for bimanual hand-object interaction. EgoTouch comprises 208 manipulation tasks spanning 1,891 episodes in diverse indoor and outdoor environments, with synchronized multi-view RGB (head-mounted egocentric and dual wrist-mounted cameras), bimanual 3D hand pose, and continuous pressure maps from wearable tactile sensors. Building on EgoTouch, we introduce TouchAnything, a baseline multi-view vision-to-touch prediction framework that uses the egocentric view as the primary input and flexibly leverages available wrist-mounted views at inference time. Experiments show that incorporating wrist-mounted views generally improves tactile prediction over egocentric-only input, achieving up to 5.0% relative improvement in Contact IoU and 6.1% relative improvement in Volumetric IoU. We will publicly release the dataset, code, and benchmark.

Related papers

• EgoTac: In-the-wild Tactile Prediction from Egocentric Vision [33.40360812855072]
  We introduce EgoTac, a model that predicts rich tactile information directly from egocentric human videos.<n>EgoTac is trained on a unified corpus of over 5.7M image-tactile pairs.<n>It consistently outperforms state-of-the-art contact estimators.
  arXiv  Detail & Related papers  (2026-08-15T06:18:13Z)
• TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation [50.608989079323784]
  We propose TouchThinker, a tactile-language framework that scales tactile commonsense reasoning to the open world from both data and representation perspectives.<n>First, we construct TouchThinker-1M, a million-scale, multi-source tactile reasoning dataset covering textbf415 objects, textbf8 scenarios, and textbf7 sensor types.<n>Then, we propose action-aware modeling mechanism to improve tactile representation efficiency and enable efficient reasoning.
  arXiv  Detail & Related papers  (2026-06-10T03:58:32Z)
• EgoTactile: Learning Grasp Pressure for Everyday Objects from Egocentric Video [82.13376072770363]
  Estimating full-hand grasp pressure from egocentric video is critical for immersive VR and robotic manipulation.<n>We introduce EgoTactile, a benchmark pairing egocentric video with full-hand pressure supervision for diverse everyday objects.<n>We also propose EgoPressureDiff, a conditional diffusion framework that adapts a large-scale pre-trained video diffusion backbone.
  arXiv  Detail & Related papers  (2026-06-08T09:20:03Z)
• OPENTOUCH: Bringing Full-Hand Touch to Real-World Interaction [93.88239833545623]
  We present OpenTouch, the first in-the-wild egocentric full-hand tactile dataset.<n>We show that tactile signals provide a compact yet powerful cue for grasp understanding.<n>We aim to advance multimodal egocentric perception, embodied learning, and contact-rich robotic manipulation.
  arXiv  Detail & Related papers  (2025-12-18T18:18:17Z)
• Grasp Like Humans: Learning Generalizable Multi-Fingered Grasping from Human Proprioceptive Sensorimotor Integration [26.351720551267846]
  Tactile and kinesthetic perceptions are crucial for human dexterous manipulation, enabling reliable grasping of objects via sensorimotor integration.<n>We propose a novel glove-mediated tactile-kinematic perception-prediction framework for grasp skill transfer from human intuitive and natural operation to robotic execution based on imitation learning.
  arXiv  Detail & Related papers  (2025-09-10T07:44:12Z)
• Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactions [110.43343503158306]
  This paper embeds the manual-assisted task into a vision-language-action framework, where the assistant provides services to the instructor following egocentric vision and commands.<n>Under this setting, we accomplish InterVLA, the first large-scale human-object-human interaction dataset with 11.4 hours and 1.2M frames of multimodal data.<n>We establish novel benchmarks on egocentric human motion estimation, interaction synthesis, and interaction prediction with comprehensive analysis.
  arXiv  Detail & Related papers  (2025-08-06T17:46:23Z)
• EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision [69.1005706608681]
  EgoPressure is a novel egocentric dataset that captures detailed touch contact and pressure interactions.<n>Our dataset comprises 5 hours of recorded interactions from 21 participants captured simultaneously by one head-mounted and seven stationary Kinect cameras.
  arXiv  Detail & Related papers  (2024-09-03T18:53:32Z)
• Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects [89.95728475983263]
  holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. We design the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks.
  arXiv  Detail & Related papers  (2024-03-25T05:12:21Z)
• Touch and Go: Learning from Human-Collected Vision and Touch [16.139106833276]
  We propose a dataset with paired visual and tactile data called Touch and Go. Human data collectors probe objects in natural environments using tactile sensors. Our dataset spans a large number of "in the wild" objects and scenes.
  arXiv  Detail & Related papers  (2022-11-22T18:59:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.