VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling

• URL: http://arxiv.org/abs/2512.02902v1
• Date: Tue, 02 Dec 2025 16:16:13 GMT
• Title: VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling
• Abstract summary: We show that vision-language-action models degrade sharply under novel camera viewpoints and visual perturbations.<n>We propose a one-shot adaptation framework that recalibrates visual representations through lightweight, learnable updates.
• Score: 60.341503853471494
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language-action (VLA) models achieve strong in-distribution performance but degrade sharply under novel camera viewpoints and visual perturbations. We show that this brittleness primarily arises from misalignment in Spatial Modeling, rather than Physical Modeling. To address this, we propose a one-shot adaptation framework that recalibrates visual representations through lightweight, learnable updates. Our first method, Feature Token Modulation (FTM), applies a global affine transformation to visual tokens and improves Libero viewpoint accuracy from 48.5% to 87.1% with only 4K parameters. Building on this, Feature Linear Adaptation (FLA) introduces low-rank updates to the ViT encoder, achieving 90.8% success with 4.7M parameters -- matching LoRA-scale finetuning at far lower cost. Together, these results reveal substantial untapped robustness in pretrained VLA models and demonstrate that targeted, minimal visual adaptation is sufficient to restore viewpoint generalization.

Related papers

• GIFT: Goal-Injected Fine-Tuning for Efficient Manipulation Policy Adaptation [24.751598161321933]
  Goal-Injected Fine-Tuning (GIFT) is a lightweight and efficient fine-tuning framework that seamlessly integrates generated goal images into pretrained VLA models.<n>Our approach introduces goal image features into observations via a zero-d convolution which progressively grows parameters from zero.<n>As training proceeds, goal information is gradually incorporated, enabling efficient goal understanding without disrupting model stability.
  arXiv  Detail & Related papers  (2026-09-07T03:56:49Z)
• Zero-Reference Joint Low-Light Enhancement and Deblurring via Visual Autoregressive Modeling with VLM-Derived Modulation [18.67176370944511]
  Real-world dark images commonly exhibit not only low visibility and contrast but also complex noise and blur, posing significant restoration challenges.<n>We propose a generative framework based on visual autoregressive ( VAR) modeling, guided by perceptual priors from the vision-language model (VLM)<n>Our framework is fully unsupervised and achieves state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2025-11-23T19:08:45Z)
• FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference [14.00844847268286]
  We introduce an imitation-learning-based Self-Speculative Decoding framework, named FastVLM, to address these limitations.<n>Our approach employs a lightweight draft model for token generation in an autoregressive manner, while a full model verifies these tokens non-autoregressively.<n>Our method speeds up the inference process by 1.55-1.85x as compared to the final layer with minimal loss in performance.
  arXiv  Detail & Related papers  (2025-10-26T11:49:20Z)
• VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models [0.18665975431697424]
  Foundation models have advanced computer vision by enabling strong performance across diverse tasks through large-scale pretraining and supervised fine-tuning.<n>We introduce a novel formulation of self-supervised fine-tuning for vision foundation models, where the model is adapted to a new domain without requiring annotations.<n>Our method is referred to as VESSA: Video-based objEct-centric Self-Supervised Adaptation for visual foundation models.
  arXiv  Detail & Related papers  (2025-10-23T20:44:28Z)
• ScaleNet: Scaling up Pretrained Neural Networks with Incremental Parameters [67.87703790962388]
  We introduce ScaleNet, an efficient approach for scaling vision transformers (ViTs)<n>Unlike conventional training from scratch, ScaleNet facilitates rapid model expansion with negligible increases in parameters.<n>We show that ScaleNet achieves a 7.42% accuracy improvement over training from scratch while requiring only one-third of the training epochs.
  arXiv  Detail & Related papers  (2025-10-21T09:07:25Z)
• Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models [64.67721492968941]
  We propose a Text-Guided Attention for Zero-Shot Robustness (TGA-ZSR) framework. Our goal is to maintain the generalization of the CLIP model and enhance its adversarial robustness. Our method yields a 9.58% enhancement in zero-shot robust accuracy over the current state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-10-29T07:15:09Z)
• Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modification [46.25272949924458]
  It is believed that fine-tuning the parameters of VLMs corrupts the pre-trained knowledge since fine-tuning the CLIP model even degrades performance. We propose ClipFit, a method to fine-tune CLIP without introducing any overhead of extra parameters. We demonstrate that ClipFit can improve the performance of zero-shot CLIP by 7.27% average harmonic mean accuracy.
  arXiv  Detail & Related papers  (2024-09-25T08:07:18Z)
• RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness [88.82621231987815]
  We introduce RLAIF-V, a novel framework that aligns MLLMs in a fully open-source paradigm.<n> RLAIF-V maximally explores open-source MLLMs from two perspectives, including high-quality feedback data generation.<n>Experiments on six benchmarks in both automatic and human evaluation show that RLAIF-V substantially enhances the trustworthiness of models.
  arXiv  Detail & Related papers  (2024-05-27T14:37:01Z)
• DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception [66.88792390480343]
  We propose DEEM, a simple but effective approach that utilizes the generative feedback of diffusion models to align the semantic distributions of the image encoder.<n>DEEM exhibits enhanced robustness and a superior capacity to alleviate model hallucinations while utilizing fewer trainable parameters, less pre-training data, and a smaller base model size.
  arXiv  Detail & Related papers  (2024-05-24T05:46:04Z)
• EMR-Merging: Tuning-Free High-Performance Model Merging [55.03509900949149]
  We show that Elect, Mask & Rescale-Merging (EMR-Merging) shows outstanding performance compared to existing merging methods. EMR-Merging is tuning-free, thus requiring no data availability or any additional training while showing impressive performance.
  arXiv  Detail & Related papers  (2024-05-23T05:25:45Z)
• Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models [32.83187649097727]
  We build a dataset of over four million multi-view image-text pairs across more than 100K objects. We design a novel fine-tuning framework named Omniview-Tuning (OVT) OVT introduces a Cross-Viewpoint Alignment objective through a minimax-like optimization strategy.
  arXiv  Detail & Related papers  (2024-04-18T12:41:33Z)
• FullLoRA: Efficiently Boosting the Robustness of Pretrained Vision Transformers [72.83770102062141]
  Vision Transformer (ViT) model has gradually become mainstream in various computer vision tasks.<n>Existing large models tend to prioritize performance during training, potentially neglecting the robustness.<n>We develop novel LNLoRA module, incorporating a learnable layer normalization before the conventional LoRA module.<n>We propose the FullLoRA framework by integrating the learnable LNLoRA modules into all key components of ViT-based models.
  arXiv  Detail & Related papers  (2024-01-03T14:08:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.