The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling

• URL: http://arxiv.org/abs/2604.03191v1
• Date: Fri, 03 Apr 2026 17:06:31 GMT
• Title: The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling
• Abstract summary: Scaling Vision-Language-Action (VLA) models by upgrading the vision encoder is expected to improve downstream manipulation performance.<n>We show that this expectation fails when actions are represented as discrete tokens.<n>In any visuomotor pipeline, scaling behavior is governed by the location of the tightest information bottleneck.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling Vision-Language-Action (VLA) models by upgrading the vision encoder is expected to improve downstream manipulation performance--as it does in vision-language modeling. We show that this expectation fails when actions are represented as discrete tokens, and explain why through an information-theoretic principle we call the Compression Gap: in any visuomotor pipeline, scaling behavior is governed by the location of the tightest information bottleneck. When actions are continuous (e.g., Diffusion Policy), the vision encoder is the binding constraint, and upgrading it directly improves performance. When actions are discretized through a fixed-capacity codebook (e.g., OAT), the codebook becomes the binding constraint, and encoder improvements cannot propagate past it--regardless of how rich the upstream representation is. We validate this principle on the LIBERO benchmark with three lines of evidence: a factorial experiment showing that encoder upgrades improve Diffusion Policy by over 21 percentage points while OAT gains are substantially attenuated across model scales; an encoder quality gradient across four encoders confirming that Diffusion Policy tracks encoder quality monotonically while OAT remains flat; and a codebook size experiment demonstrating that relaxing codebook capacity partially recovers encoder sensitivity, providing causal evidence for the bottleneck hypothesis. Our findings reveal that scaling in Physical AI requires identifying where information bottlenecks lie in the pipeline, rather than uniformly increasing model or data size.

Related papers

• Compressing AI Traffic: Standardized Neural Network Coding of Visual-Token Representations in Split Vision-Language Inference [15.357950338341285]
  Video-MME accuracy remains close to the uncompressed reference up to a 98% reduction of the transmitted BF16 tensor and only then collapses.<n>Open-ended MLVU generation shows the same plateau-and- robustness profile under an LLM judge.
  arXiv  Detail & Related papers  (2026-09-01T13:07:32Z)
• Just Noticeable Difference Modeling for Token Compression in Vision-Language-Action Models [62.522141464687195]
  We introduce Action-JND, which extends the principle of just noticeable difference (JND) to embodied perception.<n>A token change is considered tolerable only when the induced action deviation remains within a tolerated margin.<n>The resulting action-tolerance score serves as a plug-and-play criterion for VLA compression paradigms.
  arXiv  Detail & Related papers  (2026-08-21T15:59:37Z)
• Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment [15.70133983324942]
  Self-guidance is used to align the decoder's internal feature manifold when processing both the quantized tokens and their original continuous embeddings.<n>This requires minimal training overhead and no inference-time changes.<n>applied to XCodec2, self-guidance improves all reconstruction metrics, achieving state-of-the-art low-bitrate performance.
  arXiv  Detail & Related papers  (2026-06-11T06:06:02Z)
• OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence [113.73007911004446]
  OneVision-Encoder encodes video by compressing visual structure into semantic meaning.<n>Codec-aligned, patch-level sparsity is a foundational principle, enabling OV-Encoder as a scalable engine for next-generation visual generalists.
  arXiv  Detail & Related papers  (2026-02-09T14:06:17Z)
• How Much Information Can a Vision Token Hold? A Scaling Law for Recognition Limits in VLMs [14.40434528827243]
  Recent vision-centric approaches have made significant strides in long-context modeling.<n>These models encode rendered text into continuous vision tokens, achieving high compression rates without sacrificing recognition precision.<n>However, viewing the vision encoder as a lossy channel with finite representational capacity raises a fundamental question: what is the information upper bound of visual tokens?
  arXiv  Detail & Related papers  (2026-01-28T15:21:51Z)
• VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstruction [83.50898344094153]
  VQRAE produces Continuous semantic features for image understanding and tokens for visual generation within a unified tokenizer.<n>Design enables negligible semantic information for maintaining the ability of multimodal understanding, discrete tokens.<n>VQRAE presents competitive performance on several benchmarks of visual understanding, generation and reconstruction.
  arXiv  Detail & Related papers  (2025-11-28T17:26:34Z)
• Discrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies [62.653984010274485]
  Vision-Language-Action (VLA) models adapt large vision-language backbones to map images and instructions into robot actions.<n> prevailingAs either generate actions auto-regressively in a fixed left-to-right order or attach separate or diffusion heads outside the backbone.<n>We present Discrete Diffusion VLA, a unified-transformer policy that models discretized action chunks with discrete diffusion.
  arXiv  Detail & Related papers  (2025-08-27T17:39:11Z)
• NinA: Normalizing Flows in Action. Training VLA Models with Normalizing Flows [75.70583906344815]
  Diffusion models have been widely adopted as action decoders due to their ability to model complex, multimodal action distributions.<n>We present NinA, a fast and expressive alternative to diffusion-based decoders for Vision-Language-Action (VLA) models.
  arXiv  Detail & Related papers  (2025-08-23T00:02:15Z)
• Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving [74.28510044056706]
  Existing methods usually adopt the decoupled encoder-decoder paradigm. In this work, we aim to alleviate the problem by two principles. We first predict a coarse-grained future position and action based on the encoder features. Then, conditioned on the position and action, the future scene is imagined to check the ramification if we drive accordingly.
  arXiv  Detail & Related papers  (2023-05-10T15:22:02Z)
• Variance Constrained Autoencoding [0.0]
  We show that for encoders, simultaneously attempting to enforce a distribution constraint and minimising an output distortion leads to a reduction in generative and reconstruction quality. We propose the variance-constrained autoencoder (VCAE), which only enforces a variance constraint on the latent distribution. Our experiments show that VCAE improves upon Wasserstein Autoencoder and the Variational Autoencoder in both reconstruction and generative quality on MNIST and CelebA.
  arXiv  Detail & Related papers  (2020-05-08T00:50:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.