CommIN: Semantic Image Communications as an Inverse Problem with INN-Guided Diffusion Models

• URL: http://arxiv.org/abs/2310.01130v1
• Date: Mon, 2 Oct 2023 12:06:58 GMT
• Title: CommIN: Semantic Image Communications as an Inverse Problem with INN-Guided Diffusion Models
• Authors: Jiakang Chen, Di You, Deniz G\"und\"uz, Pier Luigi Dragotti
• Abstract summary: We propose CommIN, which views the recovery of high-quality source images from degraded reconstructions as an inverse problem. We show that our CommIN significantly improves the perceptual quality compared to DeepJSCC under extreme conditions.
• Score: 20.005671042281246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint source-channel coding schemes based on deep neural networks (DeepJSCC) have recently achieved remarkable performance for wireless image transmission. However, these methods usually focus only on the distortion of the reconstructed signal at the receiver side with respect to the source at the transmitter side, rather than the perceptual quality of the reconstruction which carries more semantic information. As a result, severe perceptual distortion can be introduced under extreme conditions such as low bandwidth and low signal-to-noise ratio. In this work, we propose CommIN, which views the recovery of high-quality source images from degraded reconstructions as an inverse problem. To address this, CommIN combines Invertible Neural Networks (INN) with diffusion models, aiming for superior perceptual quality. Through experiments, we show that our CommIN significantly improves the perceptual quality compared to DeepJSCC under extreme conditions and outperforms other inverse problem approaches used in DeepJSCC.

Related papers

• SC-CDM: Enhancing Quality of Image Semantic Communication with a Compact Diffusion Model [27.462224078883786]
  We propose a generative SC for wireless image transmission (denoted as SC-CDM) We aim to redesign the swin Transformer as a new backbone for efficient semantic feature extraction and compression. We further increase the Peak Signal-to-Noise Ratio (PSNR) by over 17% on top of CNN-based DeepJSCC.
  arXiv  Detail & Related papers  (2024-10-03T01:01:04Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• High Perceptual Quality Wireless Image Delivery with Denoising Diffusion Models [10.763194436114194]
  We consider the image transmission problem over a noisy wireless channel via deep learning-based joint source-channel coding (DeepJSCC) We introduce a novel scheme that utilizes the range-null space decomposition of the target image. We demonstrate significant improvements in distortion and perceptual quality of reconstructed images compared to standard DeepJSCC and the state-of-the-art generative learning-based method.
  arXiv  Detail & Related papers  (2023-09-27T16:30:59Z)
• Generative Joint Source-Channel Coding for Semantic Image Transmission [29.738666406095074]
  Joint source-channel coding (JSCC) schemes using deep neural networks (DNNs) provide promising results in wireless image transmission. We propose two novel J SCC schemes that leverage the perceptual quality of deep generative models (DGMs) for wireless image transmission.
  arXiv  Detail & Related papers  (2022-11-24T19:14:27Z)
• Image Super-resolution with An Enhanced Group Convolutional Neural Network [102.2483249598621]
  CNNs with strong learning abilities are widely chosen to resolve super-resolution problem. We present an enhanced super-resolution group CNN (ESRGCNN) with a shallow architecture. Experiments report that our ESRGCNN surpasses the state-of-the-arts in terms of SISR performance, complexity, execution speed, image quality evaluation and visual effect in SISR.
  arXiv  Detail & Related papers  (2022-05-29T00:34:25Z)
• Perceptual Learned Source-Channel Coding for High-Fidelity Image Semantic Transmission [7.692038874196345]
  In this paper, we introduce adversarial losses to optimize deep J SCC. Our new deep J SCC architecture combines encoder, wireless channel, decoder/generator, and discriminator. A user study confirms that achieving the perceptually similar end-to-end image transmission quality, the proposed method can save about 50% wireless channel bandwidth cost.
  arXiv  Detail & Related papers  (2022-05-26T03:05:13Z)
• Adaptive Information Bottleneck Guided Joint Source and Channel Coding for Image Transmission [132.72277692192878]
  An adaptive information bottleneck (IB) guided joint source and channel coding (AIB-JSCC) is proposed for image transmission. The goal of AIB-JSCC is to reduce the transmission rate while improving the image reconstruction quality. Experimental results show that AIB-JSCC can significantly reduce the required amount of transmitted data and improve the reconstruction quality.
  arXiv  Detail & Related papers  (2022-03-12T17:44:02Z)
• Over-and-Under Complete Convolutional RNN for MRI Reconstruction [57.95363471940937]
  Recent deep learning-based methods for MR image reconstruction usually leverage a generic auto-encoder architecture. We propose an Over-and-Under Complete Convolu?tional Recurrent Neural Network (OUCR), which consists of an overcomplete and an undercomplete Convolutional Recurrent Neural Network(CRNN) The proposed method achieves significant improvements over the compressed sensing and popular deep learning-based methods with less number of trainable parameters.
  arXiv  Detail & Related papers  (2021-06-16T15:56:34Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)
• Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding [7.081604594416339]
  We consider the scenario in which images are transmitted progressively in layers over time or frequency. DeepJSCC-$l$ is an innovative solution that uses convolutional autoencoders. DeepJSCC-$l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes.
  arXiv  Detail & Related papers  (2020-09-26T00:11:50Z)
• Iterative Network for Image Super-Resolution [69.07361550998318]
  Single image super-resolution (SISR) has been greatly revitalized by the recent development of convolutional neural networks (CNN) This paper provides a new insight on conventional SISR algorithm, and proposes a substantially different approach relying on the iterative optimization. A novel iterative super-resolution network (ISRN) is proposed on top of the iterative optimization.
  arXiv  Detail & Related papers  (2020-05-20T11:11:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.