MixCompress: Mixture of Experts for Variable Rate Learned Image Compression

• URL: http://arxiv.org/abs/2607.14334v1
• Date: Wed, 15 Jul 2026 19:57:15 GMT
• Title: MixCompress: Mixture of Experts for Variable Rate Learned Image Compression
• Abstract summary: MixCompress is a unified variable bit-rate (VBR) framework based on sparse structural specialization.<n>We introduce a Mixture-of-Depths (MoD) extension to dynamically scale model capacity.
• Score: 6.773101031620122
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learned image compression (LIC) is bottlenecked by the need to store independent models for each rate-distortion operating point. Existing variable bit-rate (VBR) methods aim to reduce this overhead via dense parameter modulation, but forcing a shared backbone to approximate divergent mappings causes severe feature entanglement. Specifically, low-rate smoothing gradients inherently conflict with the preservation of high-frequency textural details, leading to sub-optimal performance. To resolve this, we propose MixCompress, a unified VBR framework based on sparse structural specialization. While sparsely gated Mixture-of-Experts (MoE) routing successfully mitigates gradient conflict, it operates on a fixed computational budget. To address the increased representational demands of higher bit-rates we introduce a Mixture-of-Depths (MoD) extension to dynamically scale model capacity. Combined with Conditional Auxiliary Transforms (CAT) for dynamic sub-band energy modulation, our hierarchical framework effectively dynamically scales capacity. Extensive evaluations demonstrate that MixCompress not only matches individually optimized single-rate baselines but can even surpass them, establishing a new Pareto frontier for computationally efficient image coding.

Related papers

• Beyond Single-Dimensional Compression: The Compound Sparsity Frontier of Large Language Models [6.94102129809152]
  Large language models (LLMs) are often compressed through static parameter pruning or dynamic token-level computation.<n>We study a minimalist compound sparsity framework that first applies low-rank approximation and channel pruning to obtain a statically compressed backbone.
  arXiv  Detail & Related papers  (2026-06-29T01:33:30Z)
• MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts [37.49087932678935]
  MoECodec is a token-aware image compression framework that supports multiple downstream tasks within a single model.<n>MoECodec replaces the FFN layers in transformer-based compression model token-wise Mixture-of-Experts (MoE)
  arXiv  Detail & Related papers  (2026-06-19T01:56:25Z)
• Adaptive Targeted Dynamic Chunking for Tokenization-Free Hierarchical Model [1.283285810929198]
  Tokenization-free hierarchical models are emerging as a promising alternative to traditional Large Language Models.<n>We propose Adaptive Targeted Dynamic Chunking (ATDC), a novel byte-compression control mechanism.
  arXiv  Detail & Related papers  (2026-05-28T15:26:34Z)
• Joint Degradation-Aware Arbitrary-Scale Super-Resolution for Variable-Rate Extreme Image Compression [28.21479870337623]
  ASSR-EIC is a novel image compression framework that supports variable-rate extreme image compression.<n>We exploit the compression- and rescaling-aware diffusion prior to guide the reconstruction, yielding high fidelity and high realism restoration.
  arXiv  Detail & Related papers  (2026-03-18T06:35:53Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Error-Propagation-Free Learned Video Compression With Dual-Domain Progressive Temporal Alignment [92.57576987521107]
  We propose a novel unifiedtransform framework with dual-domain progressive temporal alignment and quality-conditioned mixture-of-expert (QCMoE)<n>QCMoE allows continuous and consistent rate control with appealing R-D performance.<n> Experimental results show that the proposed method achieves competitive R-D performance compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2025-12-11T09:14:51Z)
• Rethinking Autoregressive Models for Lossless Image Compression via Hierarchical Parallelism and Progressive Adaptation [75.58269386927076]
  Autoregressive (AR) models are often dismissed as impractical due to prohibitive computational cost.<n>This work re-thinks this paradigm, introducing a framework built on hierarchical parallelism and progressive adaptation.<n> Experiments on diverse datasets (natural, satellite, medical) validate that our method achieves new state-of-the-art compression.
  arXiv  Detail & Related papers  (2025-11-14T06:27:58Z)
• LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy [59.1298692559785]
  Key-Value ( KV) cache is crucial component in serving transformer-based autoregressive large language models (LLMs) Existing approaches to mitigate this issue include: (1) efficient attention variants integrated in upcycling stages; (2) KV cache compression at test time; and (3) KV cache compression at test time. We propose a low-rank approximation of KV weight matrices, allowing plug-in integration with existing transformer-based LLMs without model retraining. Our method is designed to function without model tuning in upcycling stages or task-specific profiling in test stages.
  arXiv  Detail & Related papers  (2024-10-04T03:10:53Z)
• Efficient Micro-Structured Weight Unification and Pruning for Neural Network Compression [56.83861738731913]
  Deep Neural Network (DNN) models are essential for practical applications, especially for resource limited devices. Previous unstructured or structured weight pruning methods can hardly truly accelerate inference. We propose a generalized weight unification framework at a hardware compatible micro-structured level to achieve high amount of compression and acceleration.
  arXiv  Detail & Related papers  (2021-06-15T17:22:59Z)
• Spatiotemporal Entropy Model is All You Need for Learned Video Compression [9.227865598115024]
  We propose a framework to compress raw-pixel frames (rather than residual images) An entropy model is used to estimate thetemporal redundancy in a latent space rather than pixel level. Experiments showed that the proposed method outperforms state-of-the-art (SOTA) performance.
  arXiv  Detail & Related papers  (2021-04-13T10:38:32Z)
• Structured Sparsification with Joint Optimization of Group Convolution and Channel Shuffle [117.95823660228537]
  We propose a novel structured sparsification method for efficient network compression. The proposed method automatically induces structured sparsity on the convolutional weights. We also address the problem of inter-group communication with a learnable channel shuffle mechanism.
  arXiv  Detail & Related papers  (2020-02-19T12:03:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.