Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification

• URL: http://arxiv.org/abs/2601.20742v1
• Date: Wed, 28 Jan 2026 16:18:20 GMT
• Title: Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification
• Authors: Xin Jin, Jinming Liu, Yuntao Wei, Junyan Lin, Zhicheng Wang, Jianguo Huang, Xudong Yang, Yanxiao Liu, Wenjun Zeng,
• Abstract summary: "Compression Tells Intelligence" is supported by research in artificial intelligence, particularly concerning (multimodal) large language models (LLMs/MLLMs)<n>This paper provides a comprehensive overview of two dominant technique families -- Visual Coding and Vision Token Technology.<n>We experimentally show a large potential of the task-oriented token developments in the more practical tasks like multimodal LLMs (MLLMs), AI-generated content (AIGC) and embodied AI.
• Score: 23.26600803714466
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: "Compression Tells Intelligence", is supported by research in artificial intelligence, particularly concerning (multimodal) large language models (LLMs/MLLMs), where compression efficiency often correlates with improved model performance and capabilities. For compression, classical visual coding based on traditional information theory has developed over decades, achieving great success with numerous international industrial standards widely applied in multimedia (e.g., image/video) systems. Except that, the recent emergingvisual token technology of generative multi-modal large models also shares a similar fundamental objective like visual coding: maximizing semantic information fidelity during the representation learning while minimizing computational cost. Therefore, this paper provides a comprehensive overview of two dominant technique families first -- Visual Coding and Vision Token Technology -- then we further unify them from the aspect of optimization, discussing the essence of compression efficiency and model performance trade-off behind. Next, based on the proposed unified formulation bridging visual coding andvisual token technology, we synthesize bidirectional insights of themselves and forecast the next-gen visual codec and token techniques. Last but not least, we experimentally show a large potential of the task-oriented token developments in the more practical tasks like multimodal LLMs (MLLMs), AI-generated content (AIGC), and embodied AI, as well as shedding light on the future possibility of standardizing a general token technology like the traditional codecs (e.g., H.264/265) with high efficiency for a wide range of intelligent tasks in a unified and effective manner.

Related papers

• ViTCoP: Accelerating Large Vision-Language Models via Visual and Textual Semantic Collaborative Pruning [8.933549837045932]
  Large Vision-Language Models incur high computational costs due to significant redundancy in their visual tokens.<n>We propose a Visual and Textual Semantic Collaborative Pruning framework (ViTCoP) that combines redundancy filtering in the vision encoder with step-wise co-pruning within the Large Language Models.
  arXiv  Detail & Related papers  (2026-01-25T12:47:30Z)
• Revisiting MLLM Token Technology through the Lens of Classical Visual Coding [16.905045322159953]
  This paper reexamines MLLM token technology, including tokenization, token compression, and token reasoning.<n>In summary, this study presents the first comprehensive and structured technology comparison of MLLM token and visual coding.
  arXiv  Detail & Related papers  (2025-08-19T02:36:44Z)
• VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models [82.05514464090172]
  Multimodal large language models (MLLMs) have significantly advanced the integration of visual and textual understanding.<n>However, their ability to generate code from multimodal inputs remains limited.<n>We introduce VisCodex, a unified framework that seamlessly merges vision and coding language models.
  arXiv  Detail & Related papers  (2025-08-13T17:00:44Z)
• LaCo: Efficient Layer-wise Compression of Visual Tokens for Multimodal Large Language Models [62.240460476785934]
  We propose LaCo (Layer-wise Visual Token Compression), a novel framework that enables effective token compression within the intermediate layers of the vision encoder.<n>LaCo introduces two core components: 1) a layer-wise pixel-shuffle mechanism that systematically merges adjacent tokens through space-to-channel transformations, and 2) a residual learning architecture with non-parametric shortcuts.
  arXiv  Detail & Related papers  (2025-07-03T03:42:54Z)
• Token Sequence Compression for Efficient Multimodal Computing [0.19116784879310028]
  The exponential growth of Large Multimodal Models (LMMs) has driven advancements in cross-modal reasoning but at significant computational costs.<n>We highlight the redundancy and inefficiency in current vision encoders, and seek to construct an adaptive compression method for multimodal data.<n>This work is a first effort towards more effective encoding and processing of high-dimensional data, and paves the way for more scalable and sustainable multimodal systems.
  arXiv  Detail & Related papers  (2025-04-24T19:11:10Z)
• Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models [51.84752285423123]
  We introduce a novel metric, $Rank_e$, to quantify the effect of prior knowledge of the vision encoder on MLLM performance.<n>We propose VisPRE (Vision Prior Remediation), a two-stage training framework that explicitly incorporates prior knowledge at the vision encoder level.<n> Experimental results demonstrate that augmenting vision encoder's prior knowledge substantially boosts the visual understanding capabilities of MLLMs.
  arXiv  Detail & Related papers  (2025-03-23T11:33:09Z)
• From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities [31.108694010274988]
  We introduce a novel image tokenizer that bridges this gap by applying the principle of Byte-Pair.<n>Unlike conventional approaches that rely on separate visual encoders, our method directly incorporates structural prior information into image tokens.<n>This innovative approach enables Transformer models to more effectively learn and reason across modalities.
  arXiv  Detail & Related papers  (2024-10-03T02:34:31Z)
• MouSi: Poly-Visual-Expert Vision-Language Models [132.58949014605477]
  This paper proposes the use of ensemble experts technique to synergize the capabilities of individual visual encoders. This technique introduces a fusion network to unify the processing of outputs from different visual experts. In our implementation, this technique significantly reduces the positional occupancy in models like SAM, from a substantial 4096 to a more efficient and manageable 64 or even down to 1.
  arXiv  Detail & Related papers  (2024-01-30T18:09:11Z)
• i-Code: An Integrative and Composable Multimodal Learning Framework [99.56065789066027]
  i-Code is a self-supervised pretraining framework where users may flexibly combine the modalities of vision, speech, and language into unified and general-purpose vector representations. The entire system is pretrained end-to-end with new objectives including masked modality unit modeling and cross-modality contrastive learning. Experimental results demonstrate how i-Code can outperform state-of-the-art techniques on five video understanding tasks and the GLUE NLP benchmark, improving by as much as 11%.
  arXiv  Detail & Related papers  (2022-05-03T23:38:50Z)
• Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics [101.35754364753409]
  Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression. This paper summarizes VCM methodology and philosophy based on existing academia and industrial efforts.
  arXiv  Detail & Related papers  (2021-10-18T12:42:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.