TacCoRL: Integrating Tactile Feedback into VLA via Simulation

• URL: http://arxiv.org/abs/2606.11743v1
• Date: Wed, 10 Jun 2026 07:20:36 GMT
• Title: TacCoRL: Integrating Tactile Feedback into VLA via Simulation
• Abstract summary: Vision-language-action (VLA) models provide strong visual, language, and action priors for robot manipulation.<n>We present TacCoRL, a scalable framework that injects Tactile feedback into VLA policies.<n>We use a real-aligned simulator as a closed-loop training environment for contact interaction.
• Score: 38.963250081631834
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models provide strong visual, language, and action priors for robot manipulation, but visual observations alone often miss the local contact state required for contact-rich tasks. We present TacCoRL, a scalable framework that injects Tactile feedback into VLA policies and improves them through sim-real Co-training and simulation-based reinforcement learning (RL), without requiring large-scale tactile pretraining or extensive real-world contact exploration. The key idea is not only adding touch as an input, but learning how contact readings should modulate action responses in near-failure states that are rare in demonstrations and risky to collect on hardware. We use a real-aligned simulator as a closed-loop training environment for contact interaction. Mixed simulated and real trajectories first warm-start tactile-conditioned actions in the pretrained policy. Reinforcement learning with verifiable task rewards then optimizes the policy using simulated contact rollouts. It reinforces tactile-conditioned actions that lead to task completion, while a supervised objective on real trajectories keeps the refined policy anchored to deployment visual, tactile, and action distributions. The resulting policy transfers directly to the real robot without privileged simulation state or online real-world RL. Across four bimanual contact-rich tasks, the final visuo-tactile policy achieves an average success rate of 72.5%, compared to baseline of 50.0%. Result videos and more details are available at https://tac-corl.github.io/

Related papers

• $N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens [0.0]
  We present a vision-tactile-language-action (VTLA) foundation model capable of fine-grained contact-rich manipulation with tactile perception and tactile-feedback control.<n>We propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement.
  arXiv  Detail & Related papers  (2026-07-26T17:58:47Z)
• OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies [12.067164216746542]
  We introduce OmniTacTune, a policy-agnostic real-world RL pipeline.<n>It adapts tactile feedback to pretrained visual policies through residual correction.<n>It generalizes across diverse contact-rich tasks, visual base policies, and tactile representations.
  arXiv  Detail & Related papers  (2026-07-04T06:11:03Z)
• TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training [79.94818813802682]
  We introduce TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation.<n>To incorporate tactile corrective supervision into VLA post-training, TACO combines knowledge-insulated tactile adaptation with advantage-conditioned training.
  arXiv  Detail & Related papers  (2026-07-03T00:23:30Z)
• ConCent: Contact-Centric Real-to-Sim-to-Real Learning from One Demonstration [17.067367964368486]
  Sim-to-real policy transfer is a promising paradigm for scaling robot manipulation without large-scale real-world data.<n>We propose a contact-centric real-to-sim-to-real RL framework that uses task-relevant contact event sequences extracted from real demonstrations.<n>Experiments on contact-rich manipulation tasks demonstrate more stable and robust sim-to-real policy transfer compared to unconstrained RL baselines.
  arXiv  Detail & Related papers  (2026-06-29T13:16:33Z)
• VibeAct: Vibration to Actions for Contact-Rich Reactive Robot Dexterity [19.20612096722837]
  VibeAct is a framework that bridges real vibrotactile sensing and simulation-based reinforcement learning.<n>In the real world, we embed piezoelectric microphones into a dexterous robot hand and collect vibro-acoustic data through teleoperation.<n>A tactile estimator learns to predict contact and slip from real microphone waveforms, while manipulation policies are trained in simulation on the same representation computed directly from simulated contacts.
  arXiv  Detail & Related papers  (2026-06-25T17:50:07Z)
• Object-Centric Residual RL for Zero-Shot Sim-to-Real VLA Enhancement [52.93267443851685]
  Vision-Language-Action (VLA) models can generalize across diverse manipulation tasks.<n>Can a reinforcement learning policy trained purely in simulation improve the robustness of real-world VLAs zero-shot?<n>We propose an object-centric residual RL framework that refines VLA actions using object poses.
  arXiv  Detail & Related papers  (2026-06-17T11:36:54Z)
• Blind Dexterous Grasping via Real2Sim2Real Tactile Policy Learning [6.509572742369941]
  We propose a framework for tactile-only blind grasping that is deployable on a physical multi-fingered robotic hand.<n>We introduce a Real2Sim tactile calibration pipeline that constructs a contact-calibrated digital-twin simulator capable of reproducing real tactile signals.<n>Second, we improve the expressiveness of sparse tactile observations using a layout-aware tactile encoder, which incorporates sensor-geometry priors through self-supervised pretraining.<n>Third, to improve generalization to unseen objects, we train object-specific reinforcement-learning experts in the calibrated simulator and aggregate their successful grasp trajectories into a tactile-conditioned Diffusion Policy.
  arXiv  Detail & Related papers  (2026-06-10T07:46:38Z)
• TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation [24.661831588991898]
  TORL-VLA is an online reinforcement learning framework that couples tactile feedback with policy refinement for contact-rich manipulation.<n>Our method introduces a tactile-derived wrench-aware VLA to predict reference actions and future wrench sequences.<n>To stabilize learning from mixed exploratory policy-generated and human-intervention data, we introduce an intervention-censored critic.
  arXiv  Detail & Related papers  (2026-06-08T11:05:05Z)
• Learning Tactile-Aware Quadrupedal Loco-Manipulation Policies [33.25194785586049]
  We present a tactile-aware loco-manipulation policy learning pipeline with a hierarchical structure.<n>First, we leverage real-world human demonstrations to train a tactile-conditioned visuotactile high-level policy.<n>Second, we perform large-scale reinforcement learning in simulation to learn a tactile-aware whole-body control policy.
  arXiv  Detail & Related papers  (2026-04-29T21:46:58Z)
• OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation [57.133721026727706]
  We present textbfOmniViTac, a large-scale visuo-tactile-action dataset comprising $21,000+$ trajectories across $86$ tasks and $100+$ objects.<n>We propose textbf OmniVTA, a world-model-based visuo-tactile manipulation framework that integrates four tightly coupled modules.
  arXiv  Detail & Related papers  (2026-03-19T17:52:42Z)
• Semantic-Contact Fields for Category-Level Generalizable Tactile Tool Manipulation [82.63833405368159]
  Generalizing tool manipulation requires both semantic planning and precise physical control.<n>We propose Semantic-Contact Fields (SCFields), a unified 3D representation fusing visual semantics with dense contact estimates.<n> Experiments on scraping, crayon drawing, and peeling demonstrate robust category-level generalization.
  arXiv  Detail & Related papers  (2026-02-14T16:05:08Z)
• Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation [58.95799126311524]
  Humans can accomplish contact-rich tasks using vision and touch, with highly reactive capabilities such as fast response to external changes and adaptive control of contact forces.<n>Existing visual imitation learning approaches rely on action chunking to model complex behaviors.<n>We introduce TactAR, a low-cost teleoperation system that provides real-time tactile feedback through Augmented Reality.
  arXiv  Detail & Related papers  (2025-03-04T18:58:21Z)
• Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrations [25.393382192511716]
  We present an immersive virtual reality teleoperation interface designed for interactive human-like manipulation on contact rich tasks. We demonstrate the complementary strengths of massively parallel RL and imitation learning, yielding robust and natural behaviors.
  arXiv  Detail & Related papers  (2022-12-05T09:37:27Z)
• COCOI: Contact-aware Online Context Inference for Generalizable Non-planar Pushing [87.7257446869134]
  General contact-rich manipulation problems are long-standing challenges in robotics. Deep reinforcement learning has shown great potential in solving robot manipulation tasks. We propose COCOI, a deep RL method that encodes a context embedding of dynamics properties online.
  arXiv  Detail & Related papers  (2020-11-23T08:20:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.