CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding

• URL: http://arxiv.org/abs/2607.29637v1
• Date: Fri, 31 Jul 2026 17:15:51 GMT
• Title: CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
• Abstract summary: CodeShrink is an adaptive visual compression framework with three components.<n>Blank-Free Rendering replaces whitespace-dependent layouts with compact layouts and explicit structural markers, removing layout-induced tokens.<n> Dominant Token Selection jointly analyzes the instruction and code image to prune task-irrelevant visual tokens during inference.
• Score: 13.113776703707634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rendering source code as images offers a promising way to reduce the input costs of Multimodal Large Language Models (MLLMs). Adjusting image resolution can trade visual token cost against content fidelity. However, resolution scaling alone overlooks two sources of inefficiency: blank regions created by line breaks and indentation, and code regions irrelevant to the current instruction. Moreover, the best compression setting varies across inputs, tasks, and models, limiting fixed-ratio strategies. We propose CodeShrink, an adaptive visual compression framework with three components. Blank-Free Rendering replaces whitespace-dependent layouts with compact layouts and explicit structural markers, removing layout-induced tokens. Adaptive Compression Configuration uses a lightweight agent trained with reinforcement learning to predict a per-input setting that balances token efficiency and readability. Dominant Token Selection jointly analyzes the instruction and code image to prune task-irrelevant visual tokens during inference. We evaluate CodeShrink on code question answering, clone detection, and code completion. CodeShrink reduces visual token use by up to 71.2\% while matching or exceeding uncompressed text-only inputs, and consistently outperforms text-based and visual compression baselines across all three tasks. These results show that combining layout compaction, adaptive configuration, and instruction-aware pruning can make multimodal code understanding more efficient. Our code is available at https://github.com/vinsontang1/CodeShrink.

Related papers

• Can Coding Agents Solve Repository-Level Issues with Rendered Code? An Exploratory Study of Visual Representations [30.48772161139388]
  We study whether rendering code as images can serve as operational context for agentic coding.<n> Rendered code consistently reduces prompt-token cost, but the savings do not increase linearly with the visual compression ratio.<n>It largely preserves end-to-end repair accuracy, but does not overcome the performance limits of the underlying model or agent architecture.
  arXiv  Detail & Related papers  (2026-08-10T08:24:19Z)
• LayoutLite: Token-Level Implicit Layout Analysis for Efficient Document OCR [10.208363125551555]
  We propose LayoutLite, a lightweight plug-and-play module for efficient document OCR.<n>It aggregates multi-layer visual representations from the vision encoder, and predicts an importance score for each visual token with a lightweight scoring network.<n>Experiments on OmniDocBench demonstrate that LayoutLite can substantially reduce visual token length and inference cost with negligible degradation in recognition quality.
  arXiv  Detail & Related papers  (2026-07-24T11:13:44Z)
• EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling [7.872065731262025]
  EvoComp is a visual token compression framework that significantly reduces token count while preserving task accuracy.<n>We show that it retains 99.3% of the original accuracy under 3x token compression and delivers up to 1.6x speedup on mobile devices.
  arXiv  Detail & Related papers  (2026-04-18T17:52:02Z)
• CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding [24.71096142371054]
  Large Language Models (LLMs) have achieved remarkable success in source code understanding.<n>As software systems grow in scale, computational efficiency has become a critical bottleneck.
  arXiv  Detail & Related papers  (2026-02-02T08:10:21Z)
• Global Context Compression with Interleaved Vision-Text Transformation [12.971394377165767]
  In this paper, we investigate global context compression, which saves tokens at both prefilling and inference stages.<n>We propose VIST2, a novel Transformer that interleaves input text chunks alongside their visual encoding.<n>With a 4$times$ compression ratio, the resulting models demonstrate significant superiority over baselines on long writing tasks.
  arXiv  Detail & Related papers  (2026-01-15T13:29:16Z)
• Switchable Token-Specific Codebook Quantization For Face Image Compression [72.44596412563503]
  We propose a Switchable Token-Specific Codebook Quantization for face image compression.<n>By recording the codebook group to which each token belongs with a small number of bits, our method can reduce the loss incurred when decreasing the size of each codebook group.<n>Our method has demonstrated its effectiveness on face recognition datasets, achieving an average accuracy of 93.51% for reconstructed images at 0.05 bpp.
  arXiv  Detail & Related papers  (2025-10-27T02:56:17Z)
• Glyph: Scaling Context Windows via Visual-Text Compression [91.20717058018745]
  Glyph is a framework that renders long texts into images and processes them with vision-language models.<n>Our method achieves 3-4x token compression while maintaining accuracy comparable to leading long-context models.<n>Under extreme compression, a 128K-context VLM could scale to handle 1M-token-level text tasks.
  arXiv  Detail & Related papers  (2025-10-20T17:58:56Z)
• VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMs [82.72388893596555]
  Multimodal Large Language Models (MLLMs) encounter significant computational and memory bottlenecks.<n>Previous token compression techniques are often constrained by rules that risk discarding critical information.<n>We reformulate token compression as a lightweight plug-and-play framework that reformulates token compression into an end-to-end learnable decision process.
  arXiv  Detail & Related papers  (2025-10-18T17:54:18Z)
• CODA: Repurposing Continuous VAEs for Discrete Tokenization [31.932323809073477]
  textbfCODA(textbfCOntinuous-to-textbfDiscrete textbfAdaptation) is a framework that decouples compression and discretization.<n>Our approach achieves a remarkable codebook utilization of 100% and notable reconstruction FID (rFID) of $mathbf0.43$ and $mathbf1.34$ for $8 times$ and $16 times$ compression on ImageNet 256$times$ 256 benchmark.
  arXiv  Detail & Related papers  (2025-03-22T12:59:00Z)
• Scalable Image Tokenization with Index Backpropagation Quantization [74.15447383432262]
  Index Backpropagation Quantization (IBQ) is a new VQ method for the joint optimization of all codebook embeddings and the visual encoder.<n>IBQ enables scalable training of visual tokenizers and, for the first time, achieves a large-scale codebook with high dimension ($256$) and high utilization.
  arXiv  Detail & Related papers  (2024-12-03T18:59:10Z)
• Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters [54.01228554126122]
  Vision Language Models (VLMs) have demonstrated strong capabilities across various visual understanding and reasoning tasks.<n>To reduce inference costs, one can either downsize the Large Language Models (LLMs) or reduce the number of input tokens needed to represent the image.<n>We take the first steps toward designing token compression algorithms tailored for high-compression settings.
  arXiv  Detail & Related papers  (2024-11-05T18:54:21Z)
• Triple-Encoders: Representations That Fire Together, Wire Together [51.15206713482718]
  Contrastive Learning is a representation learning method that encodes relative distances between utterances into the embedding space via a bi-encoder. This study introduces triple-encoders, which efficiently compute distributed utterance mixtures from these independently encoded utterances. We find that triple-encoders lead to a substantial improvement over bi-encoders, and even to better zero-shot generalization than single-vector representation models.
  arXiv  Detail & Related papers  (2024-02-19T18:06:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.