EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling

• URL: http://arxiv.org/abs/2604.17087v1
• Date: Sat, 18 Apr 2026 17:52:02 GMT
• Title: EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling
• Abstract summary: EvoComp is a visual token compression framework that significantly reduces token count while preserving task accuracy.<n>We show that it retains 99.3% of the original accuracy under 3x token compression and delivers up to 1.6x speedup on mobile devices.
• Score: 7.872065731262025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent Multimodal Large Language Models (MLLMs) have demonstrated strong performance on vision-language understanding tasks, yet their inference efficiency is often hampered by the large number of visual tokens, particularly in high-resolution or multi-image scenarios. To address this issue, we propose EvoComp, a visual token compression framework that significantly reduces token count while preserving task accuracy. EvoComp introduces a lightweight encoder-only transformer-based compressor that selects the most informative and non-redundant visual tokens by jointly considering visual and textual contexts. A core challenge lies in providing effective supervision for training the compressor. To this end, we design an evolutionary labeling strategy that searches for token subsets minimizing the MLLM's output loss, while enforcing semantic diversity through vocabulary-based token grouping. We further train the compressor using a tailored loss function combining the GHM loss to mitigate class and difficulty imbalance, and a cosine similarity regularization to encourage semantic separation between retained and discarded tokens. Extensive experiments across multiple vision-language benchmarks show that EvoComp outperforms existing methods based on attention or similarity heuristics. Notably, it retains 99.3% of the original accuracy under 3x token compression and delivers up to 1.6x speedup on mobile devices.

Related papers

• Attention-Free and Lightweight Token Reduction for Efficient Vision-Language Models [8.482327499828154]
  We propose an attention-free and lightweight token reduction framework as a plug-and-play module for Vision-Language Models.<n>Our framework preserves both important and diverse tokens to produce a compact visual representation.
  arXiv  Detail & Related papers  (2026-07-15T06:55:08Z)
• VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression [52.23010472235226]
  Vision-language models (VLMs) process large numbers of visual tokens, resulting in substantial inference and overhead latency.<n>We propose VisCo, a training-efficient self-compression framework that reuses the pretrained VLM itself as an intrinsic compressor.
  arXiv  Detail & Related papers  (2026-07-14T13:21:36Z)
• Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification [80.62512020268626]
  UniAR is a unified autoregressive framework where a single visual tokenizer serves as the key bridge between understanding and generation.<n>UniAR adapts a pretrained vision encoder with multi-level feature fusion and a lookup-free bitwise quantization scheme.<n>A diffusion-based visual decoder operates on discrete visual tokens to decode high-fidelity images.
  arXiv  Detail & Related papers  (2026-06-16T17:59:22Z)
• EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models [22.359120747133414]
  We propose a visual token compression method that estimates token importance from multi-layer evolution deviation.<n>EvoCut can retain only 11.1% of the visual tokens on LLaVA-1.5-7B while preserving 94.4% of the average performance.
  arXiv  Detail & Related papers  (2026-06-01T06:21:28Z)
• QMoP: Query Guided Mixture-of-Projector for Efficient Visual Token Compression [35.802647459700125]
  We propose a novel framework that adaptively compresses visual tokens via three collaborative branches.<n>QMoP delivers significant savings in memory, computation, and inference time.
  arXiv  Detail & Related papers  (2026-03-22T13:42:24Z)
• Adaptive-VoCo: Complexity-Aware Visual Token Compression for Vision-Language Models [19.536595270049016]
  We propose Adaptive-VoCo, a framework that augments VoCo-LLaMA with a lightweight predictor for adaptive compression.<n> Experimental results show that our method consistently outperforms fixed-rate baselines across multiple multimodal tasks.
  arXiv  Detail & Related papers  (2025-12-20T20:24:07Z)
• AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs [29.68162972167947]
  We propose an object-level token merging strategy for Adaptive Token compression.<n>Our approach averagely, utilizes only 10% tokens while achieving almost 96% of the vanilla model's performance.
  arXiv  Detail & Related papers  (2025-11-18T06:12:15Z)
• METEOR: Multi-Encoder Collaborative Token Pruning for Efficient Vision Language Models [92.37117312251755]
  We propose a progressive pruning framework, namely Multi-Encoder collaboraTivE tOken pRuning (METEOR)<n>For multi-vision encoding, we discard redundant tokens within each encoder via a rank guided collaborative token assignment strategy.<n>For multi-vision fusion, we combine the visual features from different encoders while reducing cross-encoder redundancy with cooperative pruning.
  arXiv  Detail & Related papers  (2025-07-28T13:50:53Z)
• ToDRE: Visual Token Pruning via Diversity and Task Awareness for Efficient Large Vision-Language Models [59.47738955960352]
  ToDRE is a two-stage and training-free token compression framework.<n>It achieves superior performance by pruning tokens based on token Diversity and token-task RElevance.
  arXiv  Detail & Related papers  (2025-05-24T15:47:49Z)
• SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator [65.62084602011596]
  Large Language Models (LLMs) have exhibited exceptional performance across a spectrum of natural language processing tasks.<n>We have identified a key pattern: certain seemingly meaningless separator tokens (i.e., punctuations) contribute disproportionately to attention scores compared to semantically meaningful tokens.<n>We introduce SepLLM, a plug-and-play framework that accelerates inference by compressing these segments and eliminating redundant tokens.
  arXiv  Detail & Related papers  (2024-12-16T18:58:57Z)
• Efficient Multi-modal Large Language Models via Visual Token Grouping [55.482198808206284]
  High-resolution images and videos pose a barrier to their broader adoption.<n> compressing vision tokens in MLLMs has emerged as a promising approach to reduce inference costs.<n>We introduce VisToG, a novel grouping mechanism that leverages the capabilities of pre-trained vision encoders to group similar image segments.
  arXiv  Detail & Related papers  (2024-11-26T09:36:02Z)
• Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters [54.01228554126122]
  Vision Language Models (VLMs) have demonstrated strong capabilities across various visual understanding and reasoning tasks.<n>To reduce inference costs, one can either downsize the Large Language Models (LLMs) or reduce the number of input tokens needed to represent the image.<n>We take the first steps toward designing token compression algorithms tailored for high-compression settings.
  arXiv  Detail & Related papers  (2024-11-05T18:54:21Z)
• Efficient Large Multi-modal Models via Visual Context Compression [23.966237939194514]
  We present the study on the analysis of redundancy concerning visual tokens and efficient training within large language models. Our initial experiments show that eliminating up to 70% of visual tokens at the testing stage by simply average pooling only leads to a minimal 3% reduction in visual question answering accuracy. We introduce Visual Context on the GQA benchmark, which reduces the number of visual tokens to enhance training and inference efficiency without sacrificing performance.
  arXiv  Detail & Related papers  (2024-06-28T17:57:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.