CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2605.21854v1
• Date: Thu, 21 May 2026 01:02:41 GMT
• Title: CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models
• Abstract summary: We present CrossVLA, an empirical study of cross-paradigm Vision-Language-Action (VLA) post-training.<n>Three contributions: (i) a surrogate flow-matching log-probability estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO; and (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la
• Score: 7.489020109808801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models have rapidly converged on a small set of architectural patterns: discrete-token autoregression (e.g. OpenVLA) and continuous-action flow-matching (e.g. pi-0.5). Yet preference alignment via Direct Preference Optimisation (DPO) -- the de-facto post-training step in language models -- has been studied almost exclusively on autoregressive VLAs. We present CrossVLA, an empirical study of cross-paradigm VLA post-training. Three contributions: (i) a surrogate flow-matching log-probability estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO, finding DoRA improves over OpenVLA SFT by a mean +10.4 pp across LIBERO 4-suite (600 trials, 3 seeds) -- per-suite +20.0 Object, +11.0 Long-horizon, +8.0 Goal, +2.7 Spatial -- with zero seed variance on Object (38/50 on each of 3 seeds); (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la VLA-Cache caps at a 21% acceleration ceiling -- both chunk-level and token-level cache strategies degrade success rate to 0-80% in our benchmarks. We further pretrain a multi-view + temporal projection head on 6000 LIBERO frames, achieving 99.5% k-NN recall@1 for same-task retrieval (36x over random), available as a downstream initialisation. All code, ckpts, training logs, and reproduction scripts are open at https://github.com/lz-googlefycy/vla-lab.

Related papers

• ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation [6.874305530359895]
  We introduce ForeTime-VLA, a dense pi0.5 policy that distills a future-aware, action-equivalent representation from a frozen Fast-WAM-derived teacher.<n>Online, an eight-frame history encoder predicts this target together with manipulation phase and normalized time-to-transition.<n>Test MAE decreases from 0.134119 to 0.130593 (2.63%); paired-bootstrap 95% CI: 0.82-4.48% improvement, and test L2 decreases by 3.02%, at a 2.46-2.93% latency cost.
  arXiv  Detail & Related papers  (2026-08-21T04:40:41Z)
• Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference [22.977205925809134]
  We propose Latent Bridge, a lightweight model that predicts VLM output deltas between timesteps.<n>Latent Bridge achieves 95-100% performance retention while reducing VLM calls by 50-75%, yielding 1.65-1.73x net per-episode speedup.
  arXiv  Detail & Related papers  (2026-05-04T15:37:55Z)
• SpaCeFormer: Fast Proposal-Free Open-Vocabulary 3D Instance Segmentation [82.87586897359367]
  SpaCeFormer is a proposal-free space-curve transformer that runs at 0.14 seconds per scene.<n>SpaCeFormer reaches 21x higher mask recall than prior single-view pipelines.<n>On ScanNet200 we achieve 11.1 zero-shot mAP, a 2.8x improvement over the prior best proposal-free method.
  arXiv  Detail & Related papers  (2026-04-22T09:57:57Z)
• A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model [112.9420001646428]
  Vision-Language-Action (VLA) models have emerged as a powerful paradigm for open-world robot manipulation, but their practical deployment is often constrained by cost.<n>We present A1, a fully open-source and transparent VLA framework designed for low-cost, high- throughput inference.<n>A1 achieves state-of-the-art success rates while significantly reducing inference cost.
  arXiv  Detail & Related papers  (2026-04-07T10:18:40Z)
• SnapFlow: One-Step Action Generation for Flow-Matching VLAs via Progressive Self-Distillation [18.319705014361908]
  Vision-Language-Action (VLA) models achieve state-of-the-art generalist robotic manipulation.<n>On a modern GPU, denoising alone accounts for 80% of end-to-end inference time.<n>We present SnapFlow, a plug-and-play self-distillation method that compresses multi-step denoising into a single forward pass.
  arXiv  Detail & Related papers  (2026-04-07T09:56:03Z)
• Resource-Efficient Iterative LLM-Based NAS with Feedback Memory [49.44875022114861]
  Neural Architecture Search (NAS) automates network design, but conventional methods demand substantial computational resources.<n>We propose a closed-loop pipeline leveraging large language models (LLMs) to iteratively generate, evaluate, and refine convolutional neural network architectures.
  arXiv  Detail & Related papers  (2026-03-12T16:00:22Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)
• Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoning [38.78160379823724]
  Recurrent-Depth VLA (RD-VLA) is an architecture that achieves computational adaptivity via latent iterative refinement rather than explicit token generation.<n>RD-VLA employs a recurrent, weight-tied action head that supports arbitrary inference depth with a constant memory footprint.
  arXiv  Detail & Related papers  (2026-02-08T07:21:01Z)
• IVRA: Improving Visual-Token Relations for Robot Action Policy with Training-Free Hint-Based Guidance [35.90972175891154]
  Many Vision-Language-Action (VLA) models flatten image patches into a 1D token sequence, weakening the 2D spatial cues needed for precise manipulation.<n>We introduce a lightweight, training-free method that improves spatial understanding by exploiting affinity hints already available in the model's built-in vision.
  arXiv  Detail & Related papers  (2026-01-22T18:57:13Z)
• Steering Vision-Language-Action Models as Anti-Exploration: A Test-Time Scaling Approach [78.4812458793128]
  We propose textbfTACO, a test-time-scaling framework that applies a lightweight pseudo-count estimator as a high-fidelity verifier of action chunks.<n>Our method resembles the classical anti-exploration principle in offline reinforcement learning (RL), and being gradient-free, it incurs significant computational benefits.
  arXiv  Detail & Related papers  (2025-12-02T14:42:54Z)
• Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model [62.889356203346985]
  We propose DUal-STream diffusion (DUST), a world-model augmented VLA framework that handles the modality conflict.<n>DUST achieves up to 6% gains over a standard VLA baseline and implicit world-modeling methods.<n>On real-world tasks with the Franka Research 3, DUST outperforms baselines in success rate by 13%.
  arXiv  Detail & Related papers  (2025-10-31T16:32:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.