SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization

• URL: http://arxiv.org/abs/2603.13925v1
• Date: Sat, 14 Mar 2026 12:47:04 GMT
• Title: SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization
• Abstract summary: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for robotic manipulation.<n>We propose SmoothVLA, a novel reinforcement learning fine-tuning framework that synergistically optimize task performance and motion smoothness.<n>Our work offers a scalable approach to aligning VLA models with physical-world constraints through intrinsic reward optimization.
• Score: 11.680753432998358
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language-Action (VLA) models have emerged as a powerful paradigm for robotic manipulation. However, existing post-training methods face a dilemma between stability and exploration: Supervised Fine-Tuning (SFT) is constrained by demonstration quality and lacks generalization, whereas Reinforcement Learning (RL) improves exploration but often induces erratic, jittery trajectories that violate physical constraints. To bridge this gap, we propose SmoothVLA, a novel reinforcement learning fine-tuning framework that synergistically optimizes task performance and motion smoothness. The technical core is a physics-informed hybrid reward function that integrates binary sparse task rewards with a continuous dense term derived from trajectory jerk. Crucially, this reward is intrinsic, that computing directly from policy rollouts, without requiring extrinsic environment feedback or laborious reward engineering. Leveraging the Group Relative Policy Optimization (GRPO), SmoothVLA establishes trajectory smoothness as an explicit optimization prior, guiding the model toward physically feasible and stable control. Extensive experiments on the LIBERO benchmark demonstrate that SmoothVLA outperforms standard RL by 13.8\% in smoothness and significantly surpasses SFT in generalization across diverse tasks. Our work offers a scalable approach to aligning VLA models with physical-world constraints through intrinsic reward optimization.

Related papers

• AutoNorm: Understanding Adaptive Normalization in Transformers through Differentiable Gating [0.0]
  Normalization is a critical component for stabilizing Transformer training.<n>In this paper, we investigate a key optimization challenge in differentiable normalization gating.<n>We propose AutoNorm-S (Stabilized), a training strategy that mitigates optimization instability through a gate-freezing schedule.
  arXiv  Detail & Related papers  (2026-07-12T06:12:37Z)
• Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shaping [66.25536973294726]
  Post-training methods for text-to-image (T2I) models are prone to reward hacking.<n>Super-Linear Advantage Shaping (SLAS) reshapes the local policy space.<n>SLAS consistently surpasses the DanceGRPO baseline across multiple backbones and benchmarks.
  arXiv  Detail & Related papers  (2026-05-11T17:59:25Z)
• LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoning [90.86828952599147]
  We present LaST-R1, a novel reinforcement learning framework designed to harness "latent reasoning-before-acting" policies.<n>LaST-R1 achieves a near-perfect 99.9% average success rate on the LIBERO benchmark.<n>In real-world deployments, LaST-R1 yields up to a 22.5% average improvement over SOTA supervised fine-tuning approach.
  arXiv  Detail & Related papers  (2026-04-30T17:59:52Z)
• OmniVLA-RL: A Vision-Language-Action Model with Spatial Understanding and Online RL [1.880672844596704]
  Visual-Language-Action (VLA) models represent a paradigm shift in embodied AI.<n>Existing frameworks struggle with imprecise perception, suboptimal multimodal fusion, and instability in reinforcement learning.<n>We propose OmniVLA-RL, a novel architecture that leverages a Mix-of-Transformers (MoT) design to integrate reasoning, spatial, and action experts.
  arXiv  Detail & Related papers  (2026-04-20T01:36:58Z)
• GFT: From Imitation to Reward Fine-Tuning with Unbiased Group Advantages and Dynamic Coefficient Rectification [37.24414986596345]
  Group Fine-Tuning (GFT) is a unified post-training framework for large language models.<n>GFT consistently surpasses SFT-based methods and yields policies that integrate more smoothly with subsequent RL training.
  arXiv  Detail & Related papers  (2026-04-15T15:49:58Z)
• STRONG-VLA: Decoupled Robustness Learning for Vision-Language-Action Models under Multimodal Perturbations [26.063335767640083]
  We propose STRONG-VLA, a decoupled fine-tuning framework for Vision-Language-Action (VLA) models.<n>In Stage I, the model is exposed to a curriculum of multimodal perturbations with increasing difficulty.<n>In Stage II, the model is re-aligned with clean task distributions to recover execution fidelity while preserving robustness.<n>Experiments on the LIBERO benchmark show that STRONG-VLA consistently improves task success rates across multiple VLA architectures.
  arXiv  Detail & Related papers  (2026-04-11T06:37:47Z)
• Stabilizing Policy Optimization via Logits Convexity [59.242732612484474]
  We show that the convexity of the supervised fine-tuning loss with respect to model logits plays a key role in enabling stable training.<n>Motivated by this observation, we propose Logits Convex Optimization (LCO), a simple yet effective policy optimization framework.
  arXiv  Detail & Related papers  (2026-03-01T07:40:12Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• MAESTRO: Meta-learning Adaptive Estimation of Scalarization Trade-offs for Reward Optimization [56.074760766965085]
  Group-Relative Policy Optimization has emerged as an efficient paradigm for aligning Large Language Models (LLMs)<n>We propose MAESTRO, which treats reward scalarization as a dynamic latent policy, leveraging the model's terminal hidden states as a semantic bottleneck.<n>We formulate this as a contextual bandit problem within a bi-level optimization framework, where a lightweight Conductor network co-evolves with the policy by utilizing group-relative advantages as a meta-reward signal.
  arXiv  Detail & Related papers  (2026-01-12T05:02:48Z)
• TCPO: Thought-Centric Preference Optimization for Effective Embodied Decision-making [75.29820290660065]
  This paper proposes Thought-Centric Preference Optimization ( TCPO) for effective embodied decision-making.<n>It emphasizes the alignment of the model's intermediate reasoning process, mitigating the problem of model degradation.<n>Experiments in the ALFWorld environment demonstrate an average success rate of 26.67%, achieving a 6% improvement over RL4VLM.
  arXiv  Detail & Related papers  (2025-09-10T11:16:21Z)
• AMFT: Aligning LLM Reasoners by Meta-Learning the Optimal Imitation-Exploration Balance [7.685078284407324]
  Large Language Models (LLMs) are typically fine-tuned for reasoning tasks through a two-stage pipeline Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL)<n>Recent single-stage methods attempt to unify SFT and RL using principleds, but lack a mechanism for dynamically balancing the two paradigms.<n>We introduce textbf Meta Fine-Tuning (AMFT), a novel single-stage algorithm that learns the optimal balance between SFT's implicit, path-level reward and RL's explicit, outcome-based reward.
  arXiv  Detail & Related papers  (2025-08-09T11:40:54Z)
• LeTO: Learning Constrained Visuomotor Policy with Differentiable Trajectory Optimization [1.1602089225841634]
  This paper introduces LeTO, a method for learning constrained visuomotor policy with differentiable trajectory optimization. We quantitatively evaluate LeTO in simulation and in the real robot.
  arXiv  Detail & Related papers  (2024-01-30T23:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.