Perceptual Learned Source-Channel Coding for High-Fidelity Image
Semantic Transmission
- URL: http://arxiv.org/abs/2205.13120v1
- Date: Thu, 26 May 2022 03:05:13 GMT
- Title: Perceptual Learned Source-Channel Coding for High-Fidelity Image
Semantic Transmission
- Authors: Jun Wang, Sixian Wang, Jincheng Dai, Zhongwei Si, Dekun Zhou, Kai Niu
- Abstract summary: In this paper, we introduce adversarial losses to optimize deep J SCC.
Our new deep J SCC architecture combines encoder, wireless channel, decoder/generator, and discriminator.
A user study confirms that achieving the perceptually similar end-to-end image transmission quality, the proposed method can save about 50% wireless channel bandwidth cost.
- Score: 7.692038874196345
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: As one novel approach to realize end-to-end wireless image semantic
transmission, deep learning-based joint source-channel coding (deep JSCC)
method is emerging in both deep learning and communication communities.
However, current deep JSCC image transmission systems are typically optimized
for traditional distortion metrics such as peak signal-to-noise ratio (PSNR) or
multi-scale structural similarity (MS-SSIM). But for low transmission rates,
due to the imperfect wireless channel, these distortion metrics lose
significance as they favor pixel-wise preservation. To account for human visual
perception in semantic communications, it is of great importance to develop new
deep JSCC systems optimized beyond traditional PSNR and MS-SSIM metrics. In
this paper, we introduce adversarial losses to optimize deep JSCC, which tends
to preserve global semantic information and local texture. Our new deep JSCC
architecture combines encoder, wireless channel, decoder/generator, and
discriminator, which are jointly learned under both perceptual and adversarial
losses. Our method yields human visually much more pleasing results than
state-of-the-art engineered image coded transmission systems and traditional
deep JSCC systems. A user study confirms that achieving the perceptually
similar end-to-end image transmission quality, the proposed method can save
about 50\% wireless channel bandwidth cost.
Related papers
- Joint Source-Channel Coding: Fundamentals and Recent Progress in
Practical Designs [6.059175509501795]
Joint source-channel coding (JSCC) offers an alternative end-to-end approach by optimizing compression and channel coding together.
This article provides an overview of the information theoretic foundations of J SCC, surveys practical J SCC designs over the decades, and discusses the reasons for their limited adoption in practical systems.
arXiv Detail & Related papers (2024-09-26T06:10:29Z) - Semantic Successive Refinement: A Generative AI-aided Semantic Communication Framework [27.524671767937512]
We introduce a novel Generative AI Semantic Communication (GSC) system for single-user scenarios.
At the transmitter end, it employs a joint source-channel coding mechanism based on the Swin Transformer for efficient semantic feature extraction.
At the receiver end, an advanced Diffusion Model (DM) reconstructs high-quality images from degraded signals, enhancing perceptual details.
arXiv Detail & Related papers (2024-07-31T06:08:51Z) - Deep Joint Semantic Coding and Beamforming for Near-Space Airship-Borne Massive MIMO Network [70.63240823677182]
Near-space airship-borne communication network urgently needs reliable and efficient Airship-to-X link.
This paper proposes to integrate semantic communication with massive multiple-input multiple-output (MIMO) technology.
arXiv Detail & Related papers (2024-05-30T09:46:59Z) - Joint Channel Estimation and Feedback with Masked Token Transformers in
Massive MIMO Systems [74.52117784544758]
This paper proposes an encoder-decoder based network that unveils the intrinsic frequency-domain correlation within the CSI matrix.
The entire encoder-decoder network is utilized for channel compression.
Our method outperforms state-of-the-art channel estimation and feedback techniques in joint tasks.
arXiv Detail & Related papers (2023-06-08T06:15:17Z) - Causal Semantic Communication for Digital Twins: A Generalizable
Imitation Learning Approach [74.25870052841226]
A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing, and artificial intelligence (AI) technologies to enable many connected intelligence services.
Wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints.
A novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems.
arXiv Detail & Related papers (2023-04-25T00:15:00Z) - Generative Joint Source-Channel Coding for Semantic Image Transmission [29.738666406095074]
Joint source-channel coding (JSCC) schemes using deep neural networks (DNNs) provide promising results in wireless image transmission.
We propose two novel J SCC schemes that leverage the perceptual quality of deep generative models (DGMs) for wireless image transmission.
arXiv Detail & Related papers (2022-11-24T19:14:27Z) - Semantic Communications with Discrete-time Analog Transmission: A PAPR
Perspective [11.777292228706742]
Two salient features of DeepJSCC-based semantic communications are the exploitation of semantic-aware features directly from the source signal, and the discrete-time analog transmission (DTAT) of these features.
Compared with traditional digital communications, semantic communications with DeepJSCC provide superior reconstruction performance at the receiver and graceful degradation with diminishing channel quality, but also exhibit a large peak-to-average power ratio (PAPR) in the transmitted signal.
arXiv Detail & Related papers (2022-08-17T15:07:25Z) - Adaptive Information Bottleneck Guided Joint Source and Channel Coding
for Image Transmission [132.72277692192878]
An adaptive information bottleneck (IB) guided joint source and channel coding (AIB-JSCC) is proposed for image transmission.
The goal of AIB-JSCC is to reduce the transmission rate while improving the image reconstruction quality.
Experimental results show that AIB-JSCC can significantly reduce the required amount of transmitted data and improve the reconstruction quality.
arXiv Detail & Related papers (2022-03-12T17:44:02Z) - Transformer-based SAR Image Despeckling [53.99620005035804]
We introduce a transformer-based network for SAR image despeckling.
The proposed despeckling network comprises of a transformer-based encoder which allows the network to learn global dependencies between different image regions.
Experiments show that the proposed method achieves significant improvements over traditional and convolutional neural network-based despeckling methods.
arXiv Detail & Related papers (2022-01-23T20:09:01Z) - Bandwidth-Agile Image Transmission with Deep Joint Source-Channel Coding [7.081604594416339]
We consider the scenario in which images are transmitted progressively in layers over time or frequency.
DeepJSCC-$l$ is an innovative solution that uses convolutional autoencoders.
DeepJSCC-$l$ has comparable performance with state of the art digital progressive transmission schemes in the challenging low signal-to-noise ratio (SNR) and small bandwidth regimes.
arXiv Detail & Related papers (2020-09-26T00:11:50Z) - A U-Net Based Discriminator for Generative Adversarial Networks [86.67102929147592]
We propose an alternative U-Net based discriminator architecture for generative adversarial networks (GANs)
The proposed architecture allows to provide detailed per-pixel feedback to the generator while maintaining the global coherence of synthesized images.
The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics.
arXiv Detail & Related papers (2020-02-28T11:16:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.