$N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens

• URL: http://arxiv.org/abs/2607.23782v1
• Date: Sun, 26 Jul 2026 17:58:47 GMT
• Title: $N_0$-VTLA: Scaling Vision-Tactile-Language-Action Model with Latent Tactile Tokens
• Abstract summary: We present a vision-tactile-language-action (VTLA) foundation model capable of fine-grained contact-rich manipulation with tactile perception and tactile-feedback control.<n>We propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present $N_0$-VTLA, a vision-tactile-language-action (VTLA) foundation model capable of (1) fine-grained contact-rich manipulation with tactile perception and tactile-feedback control, and (2) offline policy improvement from stored deployment data. Building on current vision-based backbones, we propose a training recipe for tactile integration consisting of visuo-tactile pre-training, staged tactile-pathway integration, and advantage-conditioned offline policy improvement. During pre-training, the policy learns broad contact priors from NeoData, our large-scale visuo-tactile robot dataset; to our knowledge, $N_0$-VTLA is the first VTLA model pretrained on tactile data at scale. During post-training, we augment the policy with a predictive tactile pathway that distills the contact patterns learned at scale into the fine motion adjustments required by downstream tactile-centric manipulation. For offline policy improvement, we introduce ALTER, an advantage-conditioned offline reinforcement learning method that converts relative progress and trajectory-event comparisons into binary advantage labels for policy training on a fixed deployment corpus, further improving task-specific learning on contact-rich skills such as deformable object manipulation. Across contact-rich benchmarks, $N_0$-VTLA outperforms strong baselines by wide margins: it wins all nine real-robot NeoReal tasks and reaches 63.8% mean success on a twenty-task simulation suite, against 44.0% for the strongest baseline. $N_0$-VTLA policies trained with ALTER reach 75-95% success on three long-horizon real-robot tasks. These results lay a foundation for versatile tactile-driven manipulation policies.

Related papers

• $N_0$-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation [0.0]
  We present a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact.<n>To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on contact-rich tasks.<n>We use NeoForce, a unified force-based tactile representation, to form a physically grounded contact signal that conditions action generation.
  arXiv  Detail & Related papers  (2026-07-26T17:58:53Z)
• TACO: TActile World Model as a Self-COrrector forScalable VLA Post-Training [79.94818813802682]
  We introduce TACO, a tactile-aware world-model-driven framework for scalable VLA post-training in contact-rich manipulation.<n>To incorporate tactile corrective supervision into VLA post-training, TACO combines knowledge-insulated tactile adaptation with advantage-conditioned training.
  arXiv  Detail & Related papers  (2026-07-03T00:23:30Z)
• T-Rex: Tactile-Reactive Dexterous Manipulation [139.71263755530654]
  We introduce a variable-rate Mixture-of-Transformers (MoT) architecture equipped with a novel temporal tactile VQ-VAE encoder.<n>We demonstrate the effectiveness of tactile-reactive policies on 12 manipulation tasks requiring delicate force control and deformable object manipulation.
  arXiv  Detail & Related papers  (2026-06-15T17:59:55Z)
• TacCoRL: Integrating Tactile Feedback into VLA via Simulation [38.963250081631834]
  Vision-language-action (VLA) models provide strong visual, language, and action priors for robot manipulation.<n>We present TacCoRL, a scalable framework that injects Tactile feedback into VLA policies.<n>We use a real-aligned simulator as a closed-loop training environment for contact interaction.
  arXiv  Detail & Related papers  (2026-06-10T07:20:36Z)
• TORL-VLA: Tactile Guided Online Reinforcement Learning for Contact-Rich Manipulation [24.661831588991898]
  TORL-VLA is an online reinforcement learning framework that couples tactile feedback with policy refinement for contact-rich manipulation.<n>Our method introduces a tactile-derived wrench-aware VLA to predict reference actions and future wrench sequences.<n>To stabilize learning from mixed exploratory policy-generated and human-intervention data, we introduce an intervention-censored critic.
  arXiv  Detail & Related papers  (2026-06-08T11:05:05Z)
• EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models [84.73890225707264]
  We present EXPO-FT, a system for stable, sample-efficient RL finetuning of pretrained VLA policies.<n>Our system achieves perfect task performance (30/30 successes) across all evaluated tasks within an average of 19.1 minutes of online robot data.<n>We release an open-source robustness with the aim of facilitating broader adoption of RL finetuning of VLA models in robotics.
  arXiv  Detail & Related papers  (2026-05-25T06:31:03Z)
• OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation [57.133721026727706]
  We present textbfOmniViTac, a large-scale visuo-tactile-action dataset comprising $21,000+$ trajectories across $86$ tasks and $100+$ objects.<n>We propose textbf OmniVTA, a world-model-based visuo-tactile manipulation framework that integrates four tightly coupled modules.
  arXiv  Detail & Related papers  (2026-03-19T17:52:42Z)
• UniVTAC: A Unified Simulation Platform for Visuo-Tactile Manipulation Data Generation, Learning, and Benchmarking [47.900334665380115]
  We propose UniVTAC, a simulation-based visuo-tactile data platform.<n>We introduce the UniVTAC encoder, a visuo-tactile encoder trained on large-scale simulation-synthesized data.<n>We present the UniVTAC Benchmark, which consists of eight representative visuo-tactile manipulation tasks.
  arXiv  Detail & Related papers  (2026-02-10T18:57:00Z)
• VLA-Touch: Enhancing Vision-Language-Action Models with Dual-Level Tactile Feedback [21.08021535027628]
  We present VLA-Touch, an approach that enhances generalist robot policies with tactile sensing.<n>Our method introduces two key innovations: (1) a pipeline that leverages a pretrained tactile-language model that provides semantic tactile feedback for high-level task planning, and (2) a diffusion-based controller that refines VLA-generated actions with tactile signals for contact-rich manipulation.
  arXiv  Detail & Related papers  (2025-07-23T07:54:10Z)
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation [62.58034332427291]
  ForceVLA is a novel end-to-end manipulation framework.<n>It treats external force sensing as a first-class modality within VLA systems.
  arXiv  Detail & Related papers  (2025-05-28T09:24:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.