Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates

• URL: http://arxiv.org/abs/2605.22061v1
• Date: Thu, 21 May 2026 06:53:03 GMT
• Title: Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates
• Abstract summary: We propose a Multimodal DIC framework, which leverages side information in a multimodal manner into the DIC paradigm.<n>Specifically, we introduce a text-to-image diffusion-based decoder on textual side information extracted from correlated images.<n>We show that MDIC achieves state-of-the-art perceptual quality at extremely low datasets.
• Score: 14.545899504614674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distributed Image Compression (DIC) is crucial for multi-view transmission, especially when operating at extremely low bitrates (< 0.1 bpp). Its core challenge is effectively utilizing side information to achieve high-quality reconstruction under strict bitrate budgets. However, existing DIC approaches struggle to exploit global context and object-level details from side information, leading to local blurring and the loss of fine details in the reconstruction. To address these limitations, we propose a Multimodal DIC framework (MDIC), which, for the first time, leverages side information in a multimodal manner into the DIC paradigm, effectively preserving fine-grained local details and enhancing global perceptual quality in reconstructed images. Specifically, we introduce a text-to-image diffusion-based decoder conditioned on textual side information extracted from correlated images to capture shared global semantics. Moreover, we design a feature-mask generator, supervised by a multimodal fine-grained alignment task, to strengthen the exploitation of visual side information. The generated mask serves two purposes: first, it guides the extraction of fine-grained details from losslessly transmitted side information to preserve the semantic consistency of reconstructed details; second, it regulates the extraction of clustered feature representations from the quantized VQ-VAE embeddings, compensating for category information lost under the extreme compression of the primary image. Extensive experiments on the widely used KITTI Stereo and Cityscapes datasets demonstrate that MDIC achieves state-of-the-art perceptual quality at extremely low bitrates.

Related papers

• ROI-based Deep Image Compression with Implicit Bit Allocation [15.62284701009422]
  Region of Interest (ROI)-based image compression has rapidly developed due to its ability to maintain high fidelity in important regions.<n>Existing compression methods apply masks to suppress background information before quantization.<n>This work proposes an efficient ROI-based deep image compression model with implicit bit allocation.
  arXiv  Detail & Related papers  (2025-11-12T02:55:18Z)
• Extremely low-bitrate Image Compression Semantically Disentangled by LMMs from a Human Perception Perspective [12.321609213934389]
  Inspired by human progressive perception mechanism, we propose a Semantically Disentangled Image Compression framework.<n>We leverage LMMs to extract essential semantic components, including overall descriptions, object detailed description, and semantic segmentation masks.<n>We propose a training-free Object Restoration model with Attention Guidance (ORAG) built on pre-trained ControlNet to restore object details conditioned by object-level text descriptions and semantic masks.
  arXiv  Detail & Related papers  (2025-03-01T08:27:11Z)
• UniUIR: Considering Underwater Image Restoration as An All-in-One Learner [62.65503609562905]
  We propose a Universal Underwater Image Restoration method, termed as UniUIR.<n>To decouple degradation-specific issues and explore the inter-correlations among various degradations in UIR task, we designed the Mamba Mixture-of-Experts module.<n>This module extracts degradation prior information in both spatial and frequency domains, and adaptively selects the most appropriate task-specific prompts.
  arXiv  Detail & Related papers  (2025-01-22T16:10:42Z)
• MGHF: Multi-Granular High-Frequency Perceptual Loss for Image Super-Resolution [0.3958317527488535]
  We propose an invertible neural network (INN)-based naive textbfMulti-textbfGranular textbfHigh-textbfFrequency (MGHF-n) perceptual loss trained on ImageNet to overcome these issues.<n>We develop a comprehensive framework with several constraints to preserve, prioritize, and regularize information across multiple perspectives.
  arXiv  Detail & Related papers  (2024-11-20T18:56:24Z)
• SSP-IR: Semantic and Structure Priors for Diffusion-based Realistic Image Restoration [20.873676111265656]
  SSP-IR aims to fully exploit semantic and structure priors from low-quality images.<n>Our method outperforms other state-of-the-art methods overall on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-07-04T04:55:14Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• Learning Enriched Features for Fast Image Restoration and Enhancement [166.17296369600774]
  This paper presents a holistic goal of maintaining spatially-precise high-resolution representations through the entire network. We learn an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Our approach achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement.
  arXiv  Detail & Related papers  (2022-04-19T17:59:45Z)
• Single Image Dehazing with An Independent Detail-Recovery Network [117.86146907611054]
  We propose a single image dehazing method with an independent Detail Recovery Network (DRN) The DRN aims to recover the dehazed image details through local and global branches respectively. Our method outperforms the state-of-the-art dehazing methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2021-09-22T02:49:43Z)
• Multi-Stage Progressive Image Restoration [167.6852235432918]
  We propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets.
  arXiv  Detail & Related papers  (2021-02-04T18:57:07Z)
• Learning Enriched Features for Real Image Restoration and Enhancement [166.17296369600774]
  convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. We present a novel architecture with the collective goals of maintaining spatially-precise high-resolution representations through the entire network. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details.
  arXiv  Detail & Related papers  (2020-03-15T11:04:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.