MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts

• URL: http://arxiv.org/abs/2606.21033v1
• Date: Fri, 19 Jun 2026 01:56:25 GMT
• Title: MoECodec: Image Compression for joint human and machine perception via Mixture-of-Experts
• Abstract summary: MoECodec is a token-aware image compression framework that supports multiple downstream tasks within a single model.<n>MoECodec replaces the FFN layers in transformer-based compression model token-wise Mixture-of-Experts (MoE)
• Score: 37.49087932678935
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image compression for machines calls for a unified codec that serves multiple downstream vision tasks. Existing approaches either adopt task-specific end-to-end designs, raising parameter and deployment overhead, or rely on transfer-based adaptations that remain externally attached and heuristic task design. A key limitation shared by both lines of work is their largely static computation pattern, which applies similar transformations across tokens despite the fact that different image regions exhibit markedly different semantic importance and complexity for machine perception. We propose MoECodec, a token-aware image compression framework that supports multiple downstream tasks within a single model. MoECodec replaces the FFN layers in transformer-based compression model token-wise Mixture-of-Experts (MoE), enabling dynamic, token-level computation conditioned on the input content and task objective. To make MoE effective in compression model, we introduce a stable routing strategy that combines expert-choice routing with spatial total variation regularization to encourage spatially coherent assignments, and we propose a lightweight expert architecture, Group Shuffle MLP (GShMLP), to control parameter growth. Extensive experiments show consistent improvement against baselines on both conventional image reconstruction and machine tasks.

Related papers

• CrossMambaTuning: Synergistic Spatial and Cross-Layer Adaptation for Machine Vision Compression [4.294434416475728]
  We propose a novel framework named CrossMambaTuning, which integrates State Space Models with cross-layer interaction mechanisms for parameter-efficient fine-tuning.<n>Specifically, we design an efficient Mamba adapter equipped with task-specific prompts and multi-scale branching to precisely capture both local features and global dependencies.<n>Experiments demonstrate that CrossMambaTuning achieves state-of-the-art (SOTA) performance on multiple machine vision tasks, reducing parameter overhead by 72% compared to SOTA methods.
  arXiv  Detail & Related papers  (2026-08-26T09:20:57Z)
• MixCompress: Mixture of Experts for Variable Rate Learned Image Compression [6.773101031620122]
  MixCompress is a unified variable bit-rate (VBR) framework based on sparse structural specialization.<n>We introduce a Mixture-of-Depths (MoD) extension to dynamically scale model capacity.
  arXiv  Detail & Related papers  (2026-07-15T19:57:15Z)
• Auto-FlexSwitch: Efficient Dynamic Model Merging via Learnable Task Vector Compression [54.12624130407679]
  We propose T-Switch, which decomposes task vectors into three compact components.<n>We then introduce Auto-Switch, a training-free merging scheme that automatically composes task vectors.<n>We present Auto-FlexSwitch, a dynamic model merging approach that supports highly efficient task vector compression.
  arXiv  Detail & Related papers  (2026-04-30T16:58:05Z)
• UniCompress: Token Compression for Unified Vision-Language Understanding and Generation [62.943173382496276]
  Unified models aim to support both understanding and generation by encoding images into discrete tokens and processing them alongside text.<n>We propose a unified token compression algorithm that significantly reduces visual token count while preserving performance on both image understanding and generation tasks.
  arXiv  Detail & Related papers  (2026-03-11T21:27:15Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• Adaptive-VoCo: Complexity-Aware Visual Token Compression for Vision-Language Models [19.536595270049016]
  We propose Adaptive-VoCo, a framework that augments VoCo-LLaMA with a lightweight predictor for adaptive compression.<n> Experimental results show that our method consistently outperforms fixed-rate baselines across multiple multimodal tasks.
  arXiv  Detail & Related papers  (2025-12-20T20:24:07Z)
• DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs [124.52164183968145]
  We present DyMU, an efficient, training-free framework that reduces the computational burden of vision-language models (VLMs)<n>Our approach comprises two key components. First, Dynamic Token Merging (DToMe) reduces the number of visual token embeddings by merging similar tokens based on image complexity.<n>Second, Virtual Token Unmerging (VTU) simulates the expected token sequence for large language models (LLMs) by efficiently reconstructing the attention dynamics of a full sequence.
  arXiv  Detail & Related papers  (2025-04-23T18:38:18Z)
• All-in-One Transferring Image Compression from Human Perception to Multi-Machine Perception [33.05779778716361]
  Existing approaches typically adapt LIC to downstream tasks in a single-task manner, which is inefficient, lacks task interaction, and results in multiple task-specific bitstreams.<n>We propose a multi-task adaptation framework that enables transferring a pre-trained base to multiple machine vision tasks through a unified model and a single training process.
  arXiv  Detail & Related papers  (2025-04-17T15:06:52Z)
• Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform [58.60004238261117]
  We propose a versatile deep image compression network based on Spatial Feature Transform (SFT arXiv:1804.02815) Our model covers a wide range of compression rates using a single model, which is controlled by arbitrary pixel-wise quality maps. The proposed framework allows us to perform task-aware image compressions for various tasks.
  arXiv  Detail & Related papers  (2021-08-21T17:30:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.