Generative Semantic Communication: Diffusion Models Beyond Bit Recovery
- URL: http://arxiv.org/abs/2306.04321v1
- Date: Wed, 7 Jun 2023 10:36:36 GMT
- Title: Generative Semantic Communication: Diffusion Models Beyond Bit Recovery
- Authors: Eleonora Grassucci, Sergio Barbarossa, Danilo Comminiello
- Abstract summary: We propose a novel generative diffusion-guided framework for semantic communication.
We reduce bandwidth usage by sending highly-compressed semantic information only.
Our results show that objects, locations, and depths are still recognizable even in the presence of extremely noisy conditions.
- Score: 19.088596386865106
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Semantic communication is expected to be one of the cores of next-generation
AI-based communications. One of the possibilities offered by semantic
communication is the capability to regenerate, at the destination side, images
or videos semantically equivalent to the transmitted ones, without necessarily
recovering the transmitted sequence of bits. The current solutions still lack
the ability to build complex scenes from the received partial information.
Clearly, there is an unmet need to balance the effectiveness of generation
methods and the complexity of the transmitted information, possibly taking into
account the goal of communication. In this paper, we aim to bridge this gap by
proposing a novel generative diffusion-guided framework for semantic
communication that leverages the strong abilities of diffusion models in
synthesizing multimedia content while preserving semantic features. We reduce
bandwidth usage by sending highly-compressed semantic information only. Then,
the diffusion model learns to synthesize semantic-consistent scenes through
spatially-adaptive normalizations from such denoised semantic information. We
prove, through an in-depth assessment of multiple scenarios, that our method
outperforms existing solutions in generating high-quality images with preserved
semantic information even in cases where the received content is significantly
degraded. More specifically, our results show that objects, locations, and
depths are still recognizable even in the presence of extremely noisy
conditions of the communication channel. The code is available at
https://github.com/ispamm/GESCO.
Related papers
- Diffusion-based Generative Multicasting with Intent-aware Semantic Decomposition [45.73113352988143]
We develop an intent-aware generative semantic multicasting framework utilizing pre-trained diffusion models.
In the proposed framework, the transmitter decomposes the source signal to multiple semantic classes based on the multi-user intent.
We show that the simulation results demonstrate significantly reduced per-user latency compared with non-generative and intent-unaware multicasting benchmarks.
arXiv Detail & Related papers (2024-11-04T17:58:54Z) - Multimodal generative semantic communication based on latent diffusion model [13.035207938169844]
This paper introduces a multimodal generative semantic communication framework named mm-GESCO.
The framework ingests streams of visible and infrared modal image data, generates fused semantic segmentation maps, and transmits them.
At the receiving end, the framework can reconstruct the original multimodal images based on the semantic maps.
arXiv Detail & Related papers (2024-08-10T06:23:41Z) - Deep Joint Semantic Coding and Beamforming for Near-Space Airship-Borne Massive MIMO Network [70.63240823677182]
Near-space airship-borne communication network urgently needs reliable and efficient Airship-to-X link.
This paper proposes to integrate semantic communication with massive multiple-input multiple-output (MIMO) technology.
arXiv Detail & Related papers (2024-05-30T09:46:59Z) - Rethinking Multi-User Semantic Communications with Deep Generative Models [30.745379375963157]
We develop a novel generative semantic communication framework tailored for multi-user scenarios.
Under this innovative perspective, OFDMA systems should not aim to transmit the largest part of information.
The thorough experimental evaluation shows the capabilities of the novel diffusion model and the effectiveness of the proposed framework.
arXiv Detail & Related papers (2024-05-16T07:43:15Z) - Semantic Entropy Can Simultaneously Benefit Transmission Efficiency and Channel Security of Wireless Semantic Communications [55.54210451136529]
We propose SemEntropy to explore semantics of data for both adaptive transmission and physical layer encryption.
We show that SemEntropy can keep the semantic accuracy remain 95% with 60% less transmission.
arXiv Detail & Related papers (2024-02-05T12:25:02Z) - Generative AI-aided Joint Training-free Secure Semantic Communications
via Multi-modal Prompts [89.04751776308656]
This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding.
In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
arXiv Detail & Related papers (2023-09-05T23:24:56Z) - Communication-Efficient Framework for Distributed Image Semantic
Wireless Transmission [68.69108124451263]
Federated learning-based semantic communication (FLSC) framework for multi-task distributed image transmission with IoT devices.
Each link is composed of a hierarchical vision transformer (HVT)-based extractor and a task-adaptive translator.
Channel state information-based multiple-input multiple-output transmission module designed to combat channel fading and noise.
arXiv Detail & Related papers (2023-08-07T16:32:14Z) - Disentangling Learnable and Memorizable Data via Contrastive Learning
for Semantic Communications [81.10703519117465]
A novel machine reasoning framework is proposed to disentangle source data so as to make it semantic-ready.
In particular, a novel contrastive learning framework is proposed, whereby instance and cluster discrimination are performed on the data.
Deep semantic clusters of highest confidence are considered learnable, semantic-rich data.
Our simulation results showcase the superiority of our contrastive learning approach in terms of semantic impact and minimalism.
arXiv Detail & Related papers (2022-12-18T12:00:12Z) - Semantic-Conditional Diffusion Networks for Image Captioning [116.86677915812508]
We propose a new diffusion model based paradigm tailored for image captioning, namely Semantic-Conditional Diffusion Networks (SCD-Net)
In SCD-Net, multiple Diffusion Transformer structures are stacked to progressively strengthen the output sentence with better visional-language alignment and linguistical coherence.
Experiments on COCO dataset demonstrate the promising potential of using diffusion models in the challenging image captioning task.
arXiv Detail & Related papers (2022-12-06T16:08:16Z) - Semantic Information Recovery in Wireless Networks [8.508198765617195]
We present an ML-based semantic communication system SINFONY.
SINFONY communicates the meaning behind multiple messages to a single receiver for semantic recovery.
Numerical results reveal a tremendous rate-normalized SNR shift up to 20 dB compared to classically designed communication systems.
arXiv Detail & Related papers (2022-04-28T09:17:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.