CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding

• URL: http://arxiv.org/abs/2607.08974v1
• Date: Thu, 09 Jul 2026 22:34:11 GMT
• Title: CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding
• Abstract summary: CLAP (Causal Language-Action Prediction) prepends each numeric action sequence with a natural-language action description.<n>We will release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale compact VLA family from a single VLM lineage.
• Score: 16.03159066719039
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control. Directly converting pretrained VLMs into VLAs with minimal architectural change offers a more transparent path to understanding how VLM capabilities transfer across model scales. The core obstacle is output-distribution mismatch: predicting actions as bare numeric token sequences moves generation away from the VLM's pretrained language distribution, degrading the capabilities we seek to preserve. To address this, we propose CLAP (Causal Language-Action Prediction), which prepends each numeric action sequence with a natural-language action description, causally conditioning precise action-token prediction on a language-action plan without modifying the backbone architecture. With single-epoch fine-tuning alone, 2B CLAP achieves 90.8% on LIBERO (+14.9 pt over VLA-0) and improves robustness on LIBERO-PRO under language, object, and spatial perturbations. We will release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale compact VLA family from a single VLM lineage, enabling controlled analysis of VLM-to-VLA capability transfer.

Related papers

• LA4VLA: Learning to Act without Seeing via Language-Action Pretraining [51.209359878543005]
  Vision-Language-Action (VLA) models are commonly pretrained on robot demonstrations by jointly mapping visual observations and language instructions to actions.<n>We propose LA4VLA, a language-action pretraining framework that enables policies to acquire language-conditioned action priors without visual observations.
  arXiv  Detail & Related papers  (2026-06-25T17:13:02Z)
• Two Bridges, One Pathway: From VLMs to Generalizable VLAs with Embodied Trajectory-Coupled Data [93.7685703383343]
  Vision-language models (VLMs) are powerful general-purpose reasoners, yet converting them into robot control policies is surprisingly difficult.<n>We argue that this gap can be bridged gradually with the right intermediate data.<n>We show that mixing task-relevant out-of-distribution ETC data with a small amount of action data enables the model to generalize to novel visual-language conditions.
  arXiv  Detail & Related papers  (2026-06-07T08:57:51Z)
• QuoVLA: Quotient Space for Vision-Language-Action Models [51.02329790939691]
  Vision-Language-Action (VLA) models commonly adapt pretrained VLM latents to robot control by mapping visual observations and language instructions to continuous actions.<n>Our textitQuotient Theory for VLA shows that pretrained VLM latents are not action-insufficient but action-sufficient.<n>We propose QuoVLA, a quotient-space framework for VLA that compresses pretrained VLM latents into action-sufficient representations.
  arXiv  Detail & Related papers  (2026-05-24T06:28:53Z)
• EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training [36.75413663296722]
  Vision-Language-Action Models (VLAs) inherit their visual and linguistic capabilities from Vision-Language Models (VLMs)<n>Most VLAs are built from off-the-shelf VLMs that are not adapted to the embodied domain, limiting their downstream performance.<n>We propose EmbodiedMidtrain to bridge the gap between VLMs and VLAs.
  arXiv  Detail & Related papers  (2026-04-21T21:40:58Z)
• RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models [58.83401587988675]
  RoboAlign trains vision-language-action models (VLAs) to translate multimodal understanding into low-level actions.<n>Our key idea is to sample action tokens via zero-shot natural language reasoning and refine this reasoning using reinforcement learning (RL) to improve action accuracy.<n>RoboAlign achieves performance improvements of 17.5%, 18.9%, and 106.6% over SFT baselines on LIBERO, CALVIN, and real-world environments, respectively.
  arXiv  Detail & Related papers  (2026-03-22T17:57:55Z)
• Observing and Controlling Features in Vision-Language-Action Models [20.193036403223903]
  Vision-Language-Action Models (VLAs) have shown remarkable progress towards embodied intelligence.<n>We introduce and analyze two main concepts: feature-observability and feature-controllability.<n>Our results show that targeted, lightweight interventions can reliably steer a robot's behavior.
  arXiv  Detail & Related papers  (2026-03-05T18:53:50Z)
• MAPS: Preserving Vision-Language Representations via Module-Wise Proximity Scheduling for Better Vision-Language-Action Generalization [30.871663465403625]
  We present MAPS, the first robust fine-tuning framework for Vision-Language-Action (VLA) models.<n>Through systematic analysis, we uncover an empirical order in which proximity constraints should be relaxed to balance stability and flexibility.<n> MAPS linearly schedules this relaxation, enabling visual encoders to stay close to their pretrained priors while action-oriented language layers adapt more freely.
  arXiv  Detail & Related papers  (2025-11-25T03:39:37Z)
• Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies [62.653984010274485]
  Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions.<n> prevailingAs either generate actions auto-regressively in a fixed left-to-right order or attach separate or diffusion heads outside the backbone.<n>We present Discrete Diffusion VLA, a unified-transformer policy that models discretized action chunks with discrete diffusion.
  arXiv  Detail & Related papers  (2025-08-27T17:39:11Z)
• VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models [84.84277196012907]
  We propose VLsI: Verbalized Layers-to-Interactions, a new VLM family in 2B and 7B model sizes.<n>We validate VLsI across ten challenging vision-language benchmarks, achieving notable performance gains (11.0% for 2B and 17.4% for 7B) over GPT-4V.
  arXiv  Detail & Related papers  (2024-12-02T18:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.