FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning

• URL: http://arxiv.org/abs/2607.24267v1
• Date: Mon, 27 Jul 2026 11:00:26 GMT
• Title: FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning
• Abstract summary: We introduce FeelWorld, a visuo-tactile world model that predicts future visual latents and three tactile states.<n>FeelWorld organizes these states hierarchically as contact state, a 3D tactile latent that encodes force-related information, and slip state.<n>To prevent irrelevant tactile signals during free-space motion from degrading visual prediction, we introduce a contact-gated asymmetric attention mechanism.
• Score: 17.259290794993895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans plan physical interactions by imagining the possible outcomes of candidate actions. However, existing visual world models primarily capture appearance dynamics while overlooking the tactile states that govern contact-rich interactions, potentially producing imagined futures that appear visually plausible but violate physical dynamics. We introduce FeelWorld, a hierarchical visuo-tactile world model that jointly predicts future visual latents and three tactile states. FeelWorld organizes these states hierarchically as contact state, a 3D tactile latent that encodes force-related information, and slip state. These states are jointly predicted by a shared latent dynamics model with explicit supervision. To prevent irrelevant tactile signals during free-space motion from degrading visual prediction, we introduce a contact-gated asymmetric attention mechanism that maintains a visual-only prediction pathway before contact and enables joint visuo-tactile dynamics prediction during contact. The model is further trained with autoregressive rollouts and context noise injection to improve robustness to compounding errors. The predicted contact and slip states also support contact-aware CEM planning. Experiments on chip grasping, fruit grasping, and USB insertion show that FeelWorld reduces 10-step LPIPS from 0.084 to 0.058 and maintains an LPIPS that is 61% lower than that of the visual baseline after an 80-step autoregressive rollout. FeelWorld also achieves an average zero-shot planning success rate of 81.7%, providing an effective approach for incorporating tactile sensing into world models.

Related papers

• HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation [16.161933557281447]
  HiTac-WAM is a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution.<n>For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates.<n>For execution, the selected tactile forecast is retained as a reference.
  arXiv  Detail & Related papers  (2026-08-20T02:28:25Z)
• AdaDexGrasp: Adaptive Dexterous Grasping via 3D Visuo-Tactile Representation Fusion [60.79507611737292]
  Humans achieve stable and adaptive grasps by seamlessly integrating visual perception and tactile feedback.<n>Existing robotic grasping approaches rely on visual inputs and lack mechanisms for tactile-guided adaptation after contact.<n>We propose a unified visuo-tactile-fusion grasping framework that integrates grasp generation, feasibility prediction, and adaptive refinement.
  arXiv  Detail & Related papers  (2026-08-06T16:29:11Z)
• TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction [15.274487606802124]
  World Action Models (WAMs) combine future-state prediction with robot action generation, but existing approaches largely rely on visual futures.<n>We present TacWAM, a mechanics-aware tactile WAM that addresses this challenge in three steps.<n>We evaluate TacWAM on four real-world contact-rich manipulation tasks covering fragile grasping, sustained surface contact, and dynamic in-hand manipulation.
  arXiv  Detail & Related papers  (2026-07-30T15:47:01Z)
• TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation [14.063238637690597]
  Dexterous manipulation in everyday environments requires both anticipation and reaction.<n>We introduce TouchWorld, a predictive-and-reactive tactile foundation model for dexterous manipulation.
  arXiv  Detail & Related papers  (2026-07-08T11:28:13Z)
• VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation [22.0330577788623]
  Contact-rich manipulation requires policies to react to local deformation, pressure, slip, and friction.<n>Existing visual-tactile policies usually feed tactile observations directly into action prediction.<n>We introduce VT-WAM, a Visual-Tactile World Action Model that jointly learns future visual prediction, tactile deformation prediction, and action prediction.
  arXiv  Detail & Related papers  (2026-07-02T17:58:36Z)
• UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Models [32.87879913150572]
  We propose a unified tactile learning framework for contact-rich manipulation.<n>We show that our method improves success rate, manipulation accuracy, and contact robustness over existing methods.
  arXiv  Detail & Related papers  (2026-06-30T14:24:00Z)
• Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention [53.84395207661823]
  We propose Tactile-WAM, a touch-aware WAM with a Tactile Asymmetric Attention Mechanism (TAAM)<n>On ManiFeel, Tactile-WAM improves the mean success rate by 38.9% overall and by 86% on contact-rich tasks.
  arXiv  Detail & Related papers  (2026-06-25T06:54:56Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAs [47.982092015932444]
  Video-Action Models (VAMs) have emerged as a promising framework for embodied intelligence.<n>We introduce the Video-Tactile Action Model (VTAM), a multimodal world modeling framework that incorporates tactile perception as a grounding signal.<n>VTAM augments a pretrained video transformer with tactile streams via a lightweight modality transfer finetuning.
  arXiv  Detail & Related papers  (2026-03-24T17:45:06Z)
• OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation [57.133721026727706]
  We present textbfOmniViTac, a large-scale visuo-tactile-action dataset comprising $21,000+$ trajectories across $86$ tasks and $100+$ objects.<n>We propose textbf OmniVTA, a world-model-based visuo-tactile manipulation framework that integrates four tightly coupled modules.
  arXiv  Detail & Related papers  (2026-03-19T17:52:42Z)
• Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation [14.221542785249524]
  We introduce DreamTacVLA, a framework that grounds VLA models in contact physics by learning to feel the future.<n>Our model adopts a hierarchical perception scheme in which high-resolution tactile images serve as micro-vision inputs.<n>To further deepen the model's understanding of fine-grained contact dynamics, we finetune the system with a tactile world model that predicts future tactile signals.
  arXiv  Detail & Related papers  (2025-12-29T21:06:33Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.