Learned Image Compression for Vision-Language-Action Models

• URL: http://arxiv.org/abs/2606.16253v1
• Date: Mon, 15 Jun 2026 05:55:58 GMT
• Title: Learned Image Compression for Vision-Language-Action Models
• Abstract summary: Vision-constrained-action (VLA) models increasingly rely on high-frequency multi-camera observations.<n>Existing image and video codecs are designed to preserve generic visual fidelity rather than the control performance of downstream VLA policies.<n>We introduce (SPatially Adaptive Rate Control) a learned image compression framework tailored for VLA-driven robots.
• Score: 12.831597450089433
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models increasingly rely on high-frequency multi-camera observations, making visual communication a major bottleneck for real-time robotic control in bandwidth-constrained or distributed deployment settings. Existing image and video codecs, however, are designed to preserve generic visual fidelity rather than the control performance of downstream VLA policies. In this work, we introduce SPARC (SPatially Adaptive Rate Control), a learned image compression framework tailored for VLA-driven robots. Our key observation is that the importance of visual information varies substantially across both camera views and spatial regions within an image. Based on this observation, SPARC employs a lightweight temporal mask selector that adaptively allocates bitrate over latent representations according to task relevance while leveraging temporal context. We further introduce a tilted rate loss that stabilizes training by reducing the tendency of entropy-based objectives to over-suppress rare yet task-critical visual patterns. Experiments on diverse robotic benchmarks, including RoboCasa365, VLABench, and LIBERO, show that SPARC consistently achieves stronger control performance than conventional image/video codecs and recent learned compression methods under the same bitrate budget. We additionally demonstrate real-world deployment benefits in remote-control settings, where our method substantially improves the bitrate-success tradeoff.

Related papers

• Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference [19.683948028719808]
  Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment.<n>We propose a system level acceleration strategy that reduces computation in both perception and action generation.<n> Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance.
  arXiv  Detail & Related papers  (2026-07-14T02:48:31Z)
• D-Compress: Detail-Preserving LiDAR Range Image Compression for Real-Time Streaming on Resource-Constrained Robots [2.6111100616898852]
  D-Compress is a new detail-preserving and fast range image compression framework for real-time streaming.<n>It outperforms state-of-the-art (SOTA) compression methods in both geometric accuracy and downstream task performance.
  arXiv  Detail & Related papers  (2026-03-14T02:15:25Z)
• AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Models [33.20237572326132]
  We propose a zero-shot camera adaptation framework without additional demonstration data, policy fine-tuning, or architectural modification.<n>Our key idea is to virtually adjust test-time camera observations to match the training camera configuration in real-time.<n>This approach consistently outperforms baselines that use data augmentation for policy fine-tuning or additional 3D-aware features for visual input.
  arXiv  Detail & Related papers  (2026-03-06T03:44:23Z)
• UCM: Unifying Camera Control and Memory with Time-aware Positional Encoding Warping for World Models [54.564740558030245]
  We present UCM, a novel framework that unifies long-term memory and precise camera control via a time-aware positional encoding warping mechanism.<n>We also introduce a scalable data curation strategy utilizing point-cloud-based rendering to simulate scene revisiting.
  arXiv  Detail & Related papers  (2026-02-26T12:54:46Z)
• Benchmarking and Enhancing VLM for Compressed Image Understanding [52.98037879935058]
  Vision-Language Models (VLMs) predominantly digest and understand high-bitrate compressed images.<n>Their ability to interpret low-bitrate compressed images has yet to be explored by far.<n>We introduce the first comprehensive benchmark to evaluate the ability of VLM against compressed images.
  arXiv  Detail & Related papers  (2025-12-24T02:59:01Z)
• MSNeRV: Neural Video Representation with Multi-Scale Feature Fusion [27.621656985302973]
  Implicit Neural representations (INRs) have emerged as a promising approach for video compression.<n>Existing INR-based methods struggle to effectively represent detail-intensive and fast-changing video content.<n>We propose a multi-scale feature fusion framework, MSNeRV, for neural video representation.
  arXiv  Detail & Related papers  (2025-06-18T08:57:12Z)
• FCA2: Frame Compression-Aware Autoencoder for Modular and Fast Compressed Video Super-Resolution [68.77813885751308]
  State-of-the-art (SOTA) compressed video super-resolution (CVSR) models face persistent challenges, including prolonged inference time, complex training pipelines, and reliance on auxiliary information.<n>We propose an efficient and scalable solution inspired by the structural and statistical similarities between hyperspectral images (HSI) and video data.<n>Our approach introduces a compression-driven dimensionality reduction strategy that reduces computational complexity, accelerates inference, and enhances the extraction of temporal information across frames.
  arXiv  Detail & Related papers  (2025-06-13T07:59:52Z)
• Plug-and-Play Versatile Compressed Video Enhancement [57.62582951699999]
  Video compression effectively reduces the size of files, making it possible for real-time cloud computing.<n>However, it comes at the cost of visual quality, challenges the robustness of downstream vision models.<n>We present a versatile-aware enhancement framework that adaptively enhance videos under different compression settings.
  arXiv  Detail & Related papers  (2025-04-21T18:39:31Z)
• VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching [23.52474883720957]
  Vision-Language-Action (VLA) models have demonstrated strong multi-modal reasoning capabilities, enabling direct action generation from visual perception and language instructions.<n>This paper introduces VLA-Cache, a training-free inference acceleration method that reduces computational overhead by adaptively caching and reusing static visual tokens across frames.
  arXiv  Detail & Related papers  (2025-02-04T09:48:14Z)
• GAN-Based Multi-View Video Coding with Spatio-Temporal EPI Reconstruction [19.919826392704472]
  We propose a novel multi-view video coding method that leverages the image generation capabilities of Generative Adrial Network (GAN) At the encoder, we construct atemporal Epipolar Plane Image (EPI) decoder and further utilize a convolutional network to extract the latent code of a GAN as Side Information (SI) At the side, we combine SI and adjacent viewpoints to reconstruct intermediate views using the GAN generator.
  arXiv  Detail & Related papers  (2022-05-07T08:52:54Z)
• Long-Short Temporal Contrastive Learning of Video Transformers [62.71874976426988]
  Self-supervised pretraining of video transformers on video-only datasets can lead to action recognition results on par or better than those obtained with supervised pretraining on large-scale image datasets. Our approach, named Long-Short Temporal Contrastive Learning, enables video transformers to learn an effective clip-level representation by predicting temporal context captured from a longer temporal extent.
  arXiv  Detail & Related papers  (2021-06-17T02:30:26Z)
• An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signal [99.49099501559652]
  Video Coding for Machine (VCM) aims to bridge the gap between visual feature compression and classical video coding. We employ a conditional deep generation network to reconstruct video frames with the guidance of learned motion pattern. By learning to extract sparse motion pattern via a predictive model, the network elegantly leverages the feature representation to generate the appearance of to-be-coded frames.
  arXiv  Detail & Related papers  (2020-01-09T14:18:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.