Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation

• URL: http://arxiv.org/abs/2606.08737v1
• Date: Sun, 07 Jun 2026 17:18:23 GMT
• Title: Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
• Abstract summary: We propose Dream-Tac, a unified Tactile-World Action Model that jointly models actions, future visual observations, and tactile dynamics.<n>Specifically, Dream-Tac introduces (i) contact-gated visuotactile fusion to selectively integrate tactile signals and (ii) a contact-aware attention bias to better regulate cross-modal interactions.<n>Across six contact-rich manipulation tasks, Dream-Tac improves action accuracy by 31.7% on average, demonstrating the effectiveness of unified visuotactile world modeling.
• Score: 40.81290792381617
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World action models inherit the predictive capability of world models, enabling action generation to be guided by anticipated future observations. However, they rely primarily on vision and often fail in contact-rich manipulation, where critical cues arise from physical interaction. In this paper, we propose Dream-Tac, a unified Tactile-World Action Model that jointly models actions, future visual observations, and tactile dynamics. Specifically, Dream-Tac introduces (i) contact-gated visuotactile fusion to selectively integrate tactile signals and (ii) a contact-aware attention bias to better regulate cross-modal interactions during manipulation. To support real-time deployment, we further design a dual-level acceleration strategy, reformulating the contact-aware bias to preserve the fused attention path during training and introducing cache-based diffusion acceleration at inference, achieving up to 2.9$\times$ faster training and 1.8$\times$ faster inference. Across six contact-rich manipulation tasks, Dream-Tac improves action accuracy by 31.7\% on average, demonstrating the effectiveness of unified visuotactile world modeling.Code is available at https://github.com/LYFCLOUDFAN/Dream-Tac.

Related papers

• DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination [39.716584408571265]
  Dexterous manipulation involves contact-rich and fine-grained interactions with the physical world.<n>We present DeCAL, a vision-language-action model that unifies understanding, imagination and action generation for contact-rich manipulation.
  arXiv  Detail & Related papers  (2026-09-08T17:47:43Z)
• LAWM-3D: Learning 3D-Aware Latent Actions from Human Videos for Generalizable Robot World Models [64.11117186468533]
  World models enable agents to perform forward rollout and planning without real-world interaction.<n>Most existing LAMs rely on single-view inputs and operate primarily in 2D pixel space.<n>We propose LAWM-3D, which introduces three tightly coupled key designs for learning 3D-aware latent actions.
  arXiv  Detail & Related papers  (2026-08-06T07:47:01Z)
• τ: Learning Touch-Augmented Vision-Language-Action Models from Future Visual Supervision [43.426795127841075]
  We present, a touch-augmented Vision-Language-Action (VLA) framework that learns an action-conditioned tactile representation from future visual supervision.<n>We also introduce TacAura, a dataset of synchronized vision, proprioception, and vision-based tactile signals across four representative contact-rich manipulation tasks.
  arXiv  Detail & Related papers  (2026-07-27T14:25:15Z)
• VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation [22.0330577788623]
  Contact-rich manipulation requires policies to react to local deformation, pressure, slip, and friction.<n>Existing visual-tactile policies usually feed tactile observations directly into action prediction.<n>We introduce VT-WAM, a Visual-Tactile World Action Model that jointly learns future visual prediction, tactile deformation prediction, and action prediction.
  arXiv  Detail & Related papers  (2026-07-02T17:58:36Z)
• VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAs [47.982092015932444]
  Video-Action Models (VAMs) have emerged as a promising framework for embodied intelligence.<n>We introduce the Video-Tactile Action Model (VTAM), a multimodal world modeling framework that incorporates tactile perception as a grounding signal.<n>VTAM augments a pretrained video transformer with tactile streams via a lightweight modality transfer finetuning.
  arXiv  Detail & Related papers  (2026-03-24T17:45:06Z)
• OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation [57.133721026727706]
  We present textbfOmniViTac, a large-scale visuo-tactile-action dataset comprising $21,000+$ trajectories across $86$ tasks and $100+$ objects.<n>We propose textbf OmniVTA, a world-model-based visuo-tactile manipulation framework that integrates four tightly coupled modules.
  arXiv  Detail & Related papers  (2026-03-19T17:52:42Z)
• Tactile Modality Fusion for Vision-Language-Action Models [22.788833830429766]
  We propose TacFiLM, a lightweight modality-fusion approach that integrates visual-tactile signals into vision-language-action (VLA) models.<n>Results show consistent improvements in success rate, direct insertion performance, completion time, and force stability across both in-distribution and out-of-distribution tasks.
  arXiv  Detail & Related papers  (2026-03-15T20:57:51Z)
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos [110.98100817695307]
  We introduce DreamDojo, a foundation world model that learns diverse interactions and dexterous controls from 44k hours of egocentric human videos.<n>Our work enables several important applications based on generative world models, including live teleoperation, policy evaluation, and model-based planning.
  arXiv  Detail & Related papers  (2026-02-06T18:49:43Z)
• Learning to Feel the Future: DreamTacVLA for Contact-Rich Manipulation [14.221542785249524]
  We introduce DreamTacVLA, a framework that grounds VLA models in contact physics by learning to feel the future.<n>Our model adopts a hierarchical perception scheme in which high-resolution tactile images serve as micro-vision inputs.<n>To further deepen the model's understanding of fine-grained contact dynamics, we finetune the system with a tactile world model that predicts future tactile signals.
  arXiv  Detail & Related papers  (2025-12-29T21:06:33Z)
• iVideoGPT: Interactive VideoGPTs are Scalable World Models [70.02290687442624]
  World models empower model-based agents to interactively explore, reason, and plan within imagined environments for real-world decision-making. This work introduces Interactive VideoGPT, a scalable autoregressive transformer framework that integrates multimodal signals--visual observations, actions, and rewards--into a sequence of tokens. iVideoGPT features a novel compressive tokenization technique that efficiently discretizes high-dimensional visual observations.
  arXiv  Detail & Related papers  (2024-05-24T05:29:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.