Perceptual Learned Source-Channel Coding for High-Fidelity Image
Semantic Transmission
- URL: http://arxiv.org/abs/2205.13120v1
- Date: Thu, 26 May 2022 03:05:13 GMT
- Title: Perceptual Learned Source-Channel Coding for High-Fidelity Image
Semantic Transmission
- Authors: Jun Wang, Sixian Wang, Jincheng Dai, Zhongwei Si, Dekun Zhou, Kai Niu
- Abstract summary: In this paper, we introduce adversarial losses to optimize deep J SCC.
Our new deep J SCC architecture combines encoder, wireless channel, decoder/generator, and discriminator.
A user study confirms that achieving the perceptually similar end-to-end image transmission quality, the proposed method can save about 50% wireless channel bandwidth cost.
- Score: 7.692038874196345
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As one novel approach to realize end-to-end wireless image semantic
transmission, deep learning-based joint source-channel coding (deep JSCC)
method is emerging in both deep learning and communication communities.
However, current deep JSCC image transmission systems are typically optimized
for traditional distortion metrics such as peak signal-to-noise ratio (PSNR) or
multi-scale structural similarity (MS-SSIM). But for low transmission rates,
due to the imperfect wireless channel, these distortion metrics lose
significance as they favor pixel-wise preservation. To account for human visual
perception in semantic communications, it is of great importance to develop new
deep JSCC systems optimized beyond traditional PSNR and MS-SSIM metrics. In
this paper, we introduce adversarial losses to optimize deep JSCC, which tends
to preserve global semantic information and local texture. Our new deep JSCC
architecture combines encoder, wireless channel, decoder/generator, and
discriminator, which are jointly learned under both perceptual and adversarial
losses. Our method yields human visually much more pleasing results than
state-of-the-art engineered image coded transmission systems and traditional
deep JSCC systems. A user study confirms that achieving the perceptually
similar end-to-end image transmission quality, the proposed method can save
about 50\% wireless channel bandwidth cost.
Related papers
- Multi-level Reliability Interface for Semantic Communications over Wireless Networks [5.9056146376982]
Joint source-channel coding (JSCC) maps source messages directly into channel input symbols.
We propose designing the source and channel mappings separately and sequentially via a novel multi-level reliability interface.
This work represents an important step towards realizing semantic communications in wireless networks.
arXiv Detail & Related papers (2024-07-07T20:15:10Z) - Deep Joint Semantic Coding and Beamforming for Near-Space Airship-Borne Massive MIMO Network [70.63240823677182]
Near-space airship-borne communication network urgently needs reliable and efficient Airship-to-X link.
This paper proposes to integrate semantic communication with massive multiple-input multiple-output (MIMO) technology.
arXiv Detail & Related papers (2024-05-30T09:46:59Z) - CommIN: Semantic Image Communications as an Inverse Problem with
INN-Guided Diffusion Models [20.005671042281246]
We propose CommIN, which views the recovery of high-quality source images from degraded reconstructions as an inverse problem.
We show that our CommIN significantly improves the perceptual quality compared to DeepJSCC under extreme conditions.
arXiv Detail & Related papers (2023-10-02T12:06:58Z) - Joint Channel Estimation and Feedback with Masked Token Transformers in
Massive MIMO Systems [74.52117784544758]
This paper proposes an encoder-decoder based network that unveils the intrinsic frequency-domain correlation within the CSI matrix.
The entire encoder-decoder network is utilized for channel compression.
Our method outperforms state-of-the-art channel estimation and feedback techniques in joint tasks.
arXiv Detail & Related papers (2023-06-08T06:15:17Z) - Causal Semantic Communication for Digital Twins: A Generalizable
Imitation Learning Approach [74.25870052841226]
A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing, and artificial intelligence (AI) technologies to enable many connected intelligence services.
Wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints.
A novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems.
arXiv Detail & Related papers (2023-04-25T00:15:00Z) - Generative Joint Source-Channel Coding for Semantic Image Transmission [29.738666406095074]
Joint source-channel coding (JSCC) schemes using deep neural networks (DNNs) provide promising results in wireless image transmission.
We propose two novel J SCC schemes that leverage the perceptual quality of deep generative models (DGMs) for wireless image transmission.
arXiv Detail & Related papers (2022-11-24T19:14:27Z) - Semantic Communications with Discrete-time Analog Transmission: A PAPR
Perspective [11.777292228706742]
Two salient features of DeepJSCC-based semantic communications are the exploitation of semantic-aware features directly from the source signal, and the discrete-time analog transmission (DTAT) of these features.
Compared with traditional digital communications, semantic communications with DeepJSCC provide superior reconstruction performance at the receiver and graceful degradation with diminishing channel quality, but also exhibit a large peak-to-average power ratio (PAPR) in the transmitted signal.
arXiv Detail & Related papers (2022-08-17T15:07:25Z) - Adaptive Information Bottleneck Guided Joint Source and Channel Coding
for Image Transmission [132.72277692192878]
An adaptive information bottleneck (IB) guided joint source and channel coding (AIB-JSCC) is proposed for image transmission.
The goal of AIB-JSCC is to reduce the transmission rate while improving the image reconstruction quality.
Experimental results show that AIB-JSCC can significantly reduce the required amount of transmitted data and improve the reconstruction quality.
arXiv Detail & Related papers (2022-03-12T17:44:02Z) - Transformer-based SAR Image Despeckling [53.99620005035804]
We introduce a transformer-based network for SAR image despeckling.
The proposed despeckling network comprises of a transformer-based encoder which allows the network to learn global dependencies between different image regions.
Experiments show that the proposed method achieves significant improvements over traditional and convolutional neural network-based despeckling methods.
arXiv Detail & Related papers (2022-01-23T20:09:01Z) - Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding [7.081604594416339]
We consider the scenario in which images are transmitted progressively in layers over time or frequency.
DeepJSCC-$l$ is an innovative solution that uses convolutional autoencoders.
DeepJSCC-$l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes.
arXiv Detail & Related papers (2020-09-26T00:11:50Z) - A U-Net Based Discriminator for Generative Adversarial Networks [86.67102929147592]
We propose an alternative U-Net based discriminator architecture for generative adversarial networks (GANs)
The proposed architecture allows to provide detailed per-pixel feedback to the generator while maintaining the global coherence of synthesized images.
The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics.
arXiv Detail & Related papers (2020-02-28T11:16:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.