TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation

• URL: http://arxiv.org/abs/2607.07287v2
• Date: Thu, 09 Jul 2026 14:55:30 GMT
• Title: TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation
• Abstract summary: Dexterous manipulation in everyday environments requires both anticipation and reaction.<n>We introduce TouchWorld, a predictive-and-reactive tactile foundation model for dexterous manipulation.
• Score: 14.063238637690597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dexterous manipulation in everyday environments requires both anticipation and reaction: a robot must predict how contact should evolve while rapidly correcting local errors caused by slip, misalignment, unstable grasping, or force mismatch. Vision and language provide semantic and geometric guidance, but they cannot reliably reveal hidden contact states such as force, slip, and contact stability. Although tactile sensing exposes these physical cues, most existing policies treat touch as a low-frequency observation stream within a monolithic action model, coupling slow task reasoning, action generation, and fast contact feedback in a single loop. We introduce TouchWorld, a predictive-and-reactive tactile foundation model for dexterous manipulation. TouchWorld uses a hierarchical policy that separates vision-language subtask planning, tactile world-model prediction, visuo-tactile goal-conditioned action generation, and high-frequency tactile residual refinement. A High-Level Planning Layer produces executable subtasks and predicts tactile subgoals; a Visuo-Tactile Goal-Conditioned Policy generates nominal action chunks; and a Tactile-Conditioned Refinement Policy performs online residual correction using recent tactile and proprioceptive feedback. By using touch as both a predictive contact reference and a fast feedback signal, TouchWorld preserves the semantic generalization of vision-language-action policies while improving local contact adaptation. Across six long-horizon and contact-rich dexterous manipulation tasks, TouchWorld achieves 65.0% success in the clean setting and 53.7% success under human perturbations, outperforming the strongest baseline by 15.7 and 18.5 percentage points, respectively.

Related papers

• AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion [60.79507611737292]
  Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback.<n>Existing robotic grasping approaches rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact.<n>We propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement.
  arXiv  Detail & Related papers  (2026-08-06T16:29:11Z)
• ReTouch: Empowering Contact-Rich Dexterous Manipulation with Online-Refined Tactile Prediction [42.34171808182058]
  We introduce ReTouch, a vision-language--action model (VLA) that supports contact-rich dexterous manipulation.<n>ReTouch builds on two main innovations for tactile representation and closed-loop action generation.
  arXiv  Detail & Related papers  (2026-08-03T07:31:10Z)
• UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models [32.87879913150572]
  We propose a unified tactile learning framework for contact-rich manipulation.<n>We show that our method improves success rate, manipulation accuracy, and contact robustness over existing methods.
  arXiv  Detail & Related papers  (2026-06-30T14:24:00Z)
• Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention [53.84395207661823]
  We propose Tactile-WAM, a touch-aware WAM with a Tactile Asymmetric Attention Mechanism (TAAM)<n>On ManiFeel, Tactile-WAM improves the mean success rate by 38.9% overall and by 86% on contact-rich tasks.
  arXiv  Detail & Related papers  (2026-06-25T06:54:56Z)
• Inference-time Policy Steering via Vision and Touch [19.707194684285223]
  Inference-time steering adapts pre-trained generative robot policies during deployment by verifying candidate actions before execution.<n>We introduce ViTaL, a visuo-tactile inference-time steering framework that formulates multimodal guidance as a bi-level optimization problem.<n>ViTaL improves overall success by 51% over the base policy, outperforms unimodal steering by at least 33%, and exceeds naive multimodal fusion by at least 20%.
  arXiv  Detail & Related papers  (2026-06-12T22:03:21Z)
• TouchThinker: Scaling Tactile Commonsense Reasoning to the Open World with Large-scale Data and Action-aware Representation [50.608989079323784]
  We propose TouchThinker, a tactile-language framework that scales tactile commonsense reasoning to the open world from both data and representation perspectives.<n>First, we construct TouchThinker-1M, a million-scale, multi-source tactile reasoning dataset covering textbf415 objects, textbf8 scenarios, and textbf7 sensor types.<n>Then, we propose action-aware modeling mechanism to improve tactile representation efficiency and enable efficient reasoning.
  arXiv  Detail & Related papers  (2026-06-10T03:58:32Z)
• TacForeSight: Force-Guided Tactile World Model for Contact-Rich Manipulation [46.82050380051203]
  Contact-rich manipulation requires robots to continuously regulate and regulate physical dynamic contact transitions or complex surface geometries.<n>Recent imitation learning methods improve contact-aware control by incorporating tactile or force feedback, but they rarely model the asymmetric roles of global force and local tactile sensing.<n>We propose TacForeSight, a lightweight force foresight framework for real-time manipulation.
  arXiv  Detail & Related papers  (2026-06-09T17:59:03Z)
• Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies [33.25194785586049]
  We present a tactile-aware loco-manipulation policy learning pipeline with a hierarchical structure.<n>First, we leverage real-world human demonstrations to train a tactile-conditioned visuotactile high-level policy.<n>Second, we perform large-scale reinforcement learning in simulation to learn a tactile-aware whole-body control policy.
  arXiv  Detail & Related papers  (2026-04-29T21:46:58Z)
• OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation [57.133721026727706]
  We present textbfOmniViTac, a large-scale visuo-tactile-action dataset comprising $21,000+$ trajectories across $86$ tasks and $100+$ objects.<n>We propose textbf OmniVTA, a world-model-based visuo-tactile manipulation framework that integrates four tightly coupled modules.
  arXiv  Detail & Related papers  (2026-03-19T17:52:42Z)
• Contact-Grounded Policy: Dexterous Visuotactile Policy with Generative Contact Grounding [5.936373185672394]
  Contact-Grounded Policy (CGP) is a visuotactile policy that grounds multi-point contacts by predicting coupled trajectories of actual robot state and tactile feedback.<n>CGP consists of two components: (i) a conditional diffusion model that forecasts future robot state and tactile feedback in a compressed latent space, and (ii) a learned contact-consistency mapping.<n>We evaluate CGP using a physical four-finger Allegro V5 hand with Digit360 fingertip tactile sensors, and a simulated five-finger Tesollo DG-5F hand with dense whole-hand tactile arrays.
  arXiv  Detail & Related papers  (2026-03-05T21:22:49Z)
• TouchGuide: Inference-Time Steering of Visuomotor Policies via Touch Guidance [53.35296919674763]
  TouchGuide is a novel cross-policy visuo-tactile fusion paradigm that fuses modalities within a low-dimensional action space.<n>touchGuide operates in two stages to guide a pre-trained diffusion or flow-matching visuomotor policy.<n>To facilitate TouchGuide training with high-quality and cost-effective data, we introduce TacUMI, a data collection system.
  arXiv  Detail & Related papers  (2026-01-28T04:22:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.