VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling

• URL: http://arxiv.org/abs/2608.08630v1
• Date: Sun, 09 Aug 2026 10:31:03 GMT
• Title: VLZip: Unified Visual and Textual Compression for Interleaved Long-Context Modeling
• Abstract summary: We introduce VLZip, a framework that unifies visual and textual compression for high-fidelity reasoning within a pure Transformer.<n>We also introduce LongVLBench, a new benchmark derived from video narratives that demands holistic, narrative-level reasoning.
• Score: 81.92510561363197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Language Models (VLMs) face significant challenges with ultra-long, interleaved image-text sequences due to the quadratic complexity of self-attention. Current solutions either resort to aggressive token pruning, risking irreversible information loss, or adopt efficient but less precise architectures, while largely ignoring the equally vital textual component. We introduce VLZip, a framework that unifies visual and textual compression for high-fidelity reasoning within a pure Transformer. At its core, VLZip hierarchically distills visual and textual segments into compact, layer-specific "soft prefixes" and injects them into each decoder layer's hidden states, drastically shortening the attention sequence while preserving fine-grained global context. To address deficient evaluations in the field, we also introduce LongVLBench, a new benchmark derived from video narratives that demands holistic, narrative-level reasoning. Extensive experiments show VLZip achieves leading performance on long-context multimodal reasoning, enabling training up to 120K tokens, a 6x increase over the baseline, and inference beyond 280K tokens with significantly reduced memory, while demonstrating the memory scalability to handle up to 2M tokens. By excelling at extreme context lengths where existing methods collapse, VLZip establishes an efficient and powerful new standard for long-context multimodal AI. Code is available at https://github.com/ShareLab-SII/VLZip.

Related papers

• PACE: A Unified Condense-and-Extract Paradigm for Fast VLM Inference [15.233755808671871]
  Existing visual token pruning methods exhibit two fundamental limitations.<n>Most approaches operate exclusively post-vision encoder, leaving the substantial latency of the visual encoding phase unoptimized.<n>We propose PACE, a framework that accelerates both the vision encoder and the Large Language Model.
  arXiv  Detail & Related papers  (2026-08-27T14:52:09Z)
• VoxZip: Semantic-Anchored Temporal KV Cache Compression for Long-Context Audio Inference [53.883871460875234]
  VoxZip is a semantic-anchored KV cache compression framework.<n>First stage uses automatic speech recognition (ASR) transcriptions as explicit semantic anchors to temporally align, compress, and fuse audio tokens.<n>Second stage employs a dynamic filtering strategy based on temporally decayed accumulated attention to evict non-essential tokens.
  arXiv  Detail & Related papers  (2026-08-09T08:17:18Z)
• BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection [21.177715869071793]
  Existing compression methods often suffer from high training costs or semantic fragmentation due to aggressive token pruning.<n>We propose BEAVER, a novel training-free framework that shifts compression from linear token removal to structure-aware hierarchical selection.<n>We show that BEAVER achieves comparable performance to state-of-the-art (SOTA) methods like LongLLMLingua.
  arXiv  Detail & Related papers  (2026-03-20T04:35:04Z)
• Seeing the Forest and the Trees: Query-Aware Tokenizer for Long-Video Multimodal Language Models [24.875526594002434]
  We present QTSplus, a visual token selection module for long video understanding scenarios.<n>It is integrated into Qwen2.5-VL and compresses the vision stream by up to textbf89% and reduces end-to-end latency by textbf28% on long videos.<n>Results show that QTSplus is an effective, general mechanism for scaling MLLMs to realworld long-video scenarios.
  arXiv  Detail & Related papers  (2025-11-14T22:41:27Z)
• Glyph: Scaling Context Windows via Visual-Text Compression [91.20717058018745]
  Glyph is a framework that renders long texts into images and processes them with vision-language models.<n>Our method achieves 3-4x token compression while maintaining accuracy comparable to leading long-context models.<n>Under extreme compression, a 128K-context VLM could scale to handle 1M-token-level text tasks.
  arXiv  Detail & Related papers  (2025-10-20T17:58:56Z)
• SparseVILA: Decoupling Visual Sparsity for Efficient VLM Inference [49.84148668264725]
  We present SparseVILA, a new paradigm for efficient VLM inference that decouples visual sparsity across the prefilling and decoding stages.<n>Built on an AWQ-optimized inference pipeline, SparseVILA achieves up to 4.0 times faster prefilling, 2.5 times faster decoding, and an overall 2.6 times end-to-end speedup on long-context video tasks.
  arXiv  Detail & Related papers  (2025-10-20T17:35:47Z)
• LongCodeZip: Compress Long Context for Code Language Models [16.940525379087326]
  LongCodeZip is a novel plug-and-play code compression framework designed specifically for Large Language Models (LLMs)<n>By effectively reducing context size while preserving essential information, LongCodeZip enables LLMs to better scale to real-world, large-scale code scenarios.
  arXiv  Detail & Related papers  (2025-10-01T02:54:57Z)
• HiTVideo: Hierarchical Tokenizers for Enhancing Text-to-Video Generation with Autoregressive Large Language Models [63.65066762436074]
  HiTVideo aims to address the potential limitations of existing video tokenizers in text-to-video generation tasks.<n>It utilizes a 3D causal VAE with a multi-layer discrete token framework, encoding video content into hierarchically structured codebooks.
  arXiv  Detail & Related papers  (2025-03-14T15:36:39Z)
• Vision-centric Token Compression in Large Language Model [51.92055188780033]
  Vision Centric Token Compression (Vist) is a slow-fast compression framework that mirrors human reading.<n>On eleven in-context learning benchmarks, Vist achieves the same accuracy with 2.3 times fewer tokens, cutting FLOPs by 16% and memory by 50%.
  arXiv  Detail & Related papers  (2025-02-02T13:10:06Z)
• Global Compression Commander: Plug-and-Play Inference Acceleration for High-Resolution Large Vision-Language Models [21.36437021964681]
  "Global Compression Commander" is a novel token compression framework for HR-LVLMs.<n>GlobalCom$2$ maintains over 90% performance while compressing 90% visual tokens, reducing FLOPs and peak memory to 9.1% and 60%.
  arXiv  Detail & Related papers  (2025-01-09T11:57:58Z)
• TreeKV: Smooth Key-Value Cache Compression with Tree Structures [19.06842704338332]
  TreeKV is a training-free method that employs a tree structure for smooth cache compression.<n>It consistently surpasses all baseline models in language modeling tasks on PG19 and OpenWebText2.
  arXiv  Detail & Related papers  (2025-01-09T06:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.