Compressing AI Traffic: Standardized Neural Network Coding of Visual-Token Representations in Split Vision-Language Inference

• URL: http://arxiv.org/abs/2609.01200v1
• Date: Tue, 01 Sep 2026 13:07:32 GMT
• Title: Compressing AI Traffic: Standardized Neural Network Coding of Visual-Token Representations in Split Vision-Language Inference
• Abstract summary: Video-MME accuracy remains close to the uncompressed reference up to a 98% reduction of the transmitted BF16 tensor and only then collapses.<n>Open-ended MLVU generation shows the same plateau-and- robustness profile under an LLM judge.
• Score: 15.357950338341285
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When the visual encoder and the language decoder of a vision-language model (VLM) run on different compute nodes, the intermediate visual-token embeddings become a communicated payload rather than an internal activation. We call such machine-consumed intermediate tensors AI traffic and ask how far they can be compressed with a standardized, training-free codec. We insert ISO/IEC 15938-17 Neural Network Coding (NNC) round trips on the complete visual interface of a Qwen3-VL-8B-Instruct video question answering pipeline, comprising the main visual-token representation and the DeepStack feature streams, while leaving weights, prompts, and generation untouched, and sweep the quantization parameter (QP) over a wide rate range. Closed-ended Video-MME accuracy remains close to the uncompressed reference up to a 98% reduction of the transmitted BF16 tensor and only then collapses; open-ended MLVU generation shows the same plateau-and-collapse profile under an LLM judge. This robustness is not due to near-lossless reconstruction: the decoded tensor is heavily discretized, carries substantial row-wise relative L2 error, and has a visibly steeper singular-value decay than its source. Downstream reasoning therefore depends on coarse structure and relative geometry rather than exact floating-point values, which argues for rate-task rather than rate-distortion optimization of AI traffic codecs.

Related papers

• PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference [15.233755808671871]
  Existing visual token pruning methods exhibit two fundamental limitations.<n>Most approaches operate exclusively post-vision encoder, leaving the substantial latency of the visual encoding phase unoptimized.<n>We propose PACE, a framework that accelerates both the vision encoder and the Large Language Model.
  arXiv  Detail & Related papers  (2026-08-27T14:52:09Z)
• Visual Token Codec: Unleashing Spatial Redundancy for ViT Feature Coding [49.540751364954666]
  Visual Token Codec separates global and patch tokens into dedicated coding paths.<n>We show that VTC consistently outperforms representative ViT feature coding baselines on classification, segmentation, and detection tasks.
  arXiv  Detail & Related papers  (2026-08-09T17:32:04Z)
• Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment [15.70133983324942]
  Self-guidance is used to align the decoder's internal feature manifold when processing both the quantized tokens and their original continuous embeddings.<n>This requires minimal training overhead and no inference-time changes.<n>applied to XCodec2, self-guidance improves all reconstruction metrics, achieving state-of-the-art low-bitrate performance.
  arXiv  Detail & Related papers  (2026-06-11T06:06:02Z)
• MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention [53.80990122465395]
  We show how a cross-attention backbone is better suited to real-time vision-language fusion than the prevailing decoder-only design.<n>Our model trails the strong Qwen2.5-VL-7B baseline overall.<n>On a single H200 with 256 frames per video, achieves it about a 5x speedup in time to first token.
  arXiv  Detail & Related papers  (2026-06-01T09:07:15Z)
• Transcoders Trace Visual Grounding and Hallucinations in Vision-Language Models [7.425292797106273]
  Generative Vision-Language Models (VLMs) perform well on multimodal reasoning, but how visual inputs are transformed to text remains poorly understood.<n>We adopt a function-centric framework based on Transcoders, sparse approximations of sublayers that act as a causal proxy for layer-wise.<n>We show that function-centric circuit decomposition yields interpretable and predictive accounts of multimodal computation inVLMs.
  arXiv  Detail & Related papers  (2026-05-21T17:34:39Z)
• The Compression Gap: Why Discrete Tokenization Limits Vision-Language-Action Model Scaling [0.0]
  Scaling Vision-Language-Action (VLA) models by upgrading the vision encoder is expected to improve downstream manipulation performance.<n>We show that this expectation fails when actions are represented as discrete tokens.<n>In any visuomotor pipeline, scaling behavior is governed by the location of the tightest information bottleneck.
  arXiv  Detail & Related papers  (2026-04-03T17:06:31Z)
• Image Coding for Machines via Feature-Preserving Rate-Distortion Optimization [31.210700220124192]
  We show that an approach to reduce the effect of compression for a given task loss is to perform rate-distortion (RDO) using the distance between features.<n>We simplify the RDO formulation to make the distortion term computable using block-based encoders.<n>We show that simulations with transformed HEVC across multiple feature extractors and downstream networks show up to 17 % bit-rate savings for the same task accuracy.
  arXiv  Detail & Related papers  (2025-04-03T02:11:26Z)
• Boosting Neural Representations for Videos with a Conditional Decoder [28.073607937396552]
  Implicit neural representations (INRs) have emerged as a promising approach for video storage and processing. This paper introduces a universal boosting framework for current implicit video representation approaches.
  arXiv  Detail & Related papers  (2024-02-28T08:32:19Z)
• Progressive Fourier Neural Representation for Sequential Video Compilation [75.43041679717376]
  Motivated by continual learning, this work investigates how to accumulate and transfer neural implicit representations for multiple complex video data over sequential encoding sessions. We propose a novel method, Progressive Fourier Neural Representation (PFNR), that aims to find an adaptive and compact sub-module in Fourier space to encode videos in each training session. We validate our PFNR method on the UVG8/17 and DAVIS50 video sequence benchmarks and achieve impressive performance gains over strong continual learning baselines.
  arXiv  Detail & Related papers  (2023-06-20T06:02:19Z)
• Reducing Redundancy in the Bottleneck Representation of the Autoencoders [98.78384185493624]
  Autoencoders are a type of unsupervised neural networks, which can be used to solve various tasks. We propose a scheme to explicitly penalize feature redundancies in the bottleneck representation. We tested our approach across different tasks: dimensionality reduction using three different dataset, image compression using the MNIST dataset, and image denoising using fashion MNIST.
  arXiv  Detail & Related papers  (2022-02-09T18:48:02Z)
• A Coding Framework and Benchmark towards Low-Bitrate Video Understanding [63.05385140193666]
  We propose a traditional-neural mixed coding framework that takes advantage of both traditional codecs and neural networks (NNs) The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved. We build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach.
  arXiv  Detail & Related papers  (2022-02-06T16:29:15Z)
• Scalable Visual Transformers with Hierarchical Pooling [61.05787583247392]
  We propose a Hierarchical Visual Transformer (HVT) which progressively pools visual tokens to shrink the sequence length. It brings a great benefit by scaling dimensions of depth/width/resolution/patch size without introducing extra computational complexity. Our HVT outperforms the competitive baselines on ImageNet and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2021-03-19T03:55:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.