Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Models

• URL: http://arxiv.org/abs/2403.17256v2
• Date: Sat, 13 Jul 2024 23:46:49 GMT
• Title: Latency-Aware Generative Semantic Communications with Pre-Trained Diffusion Models
• Authors: Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Chuan Heng Foh, Pei Xiao, Mehdi Bennis,
• Abstract summary: We develop a latency-aware semantic communications framework with pre-trained generative models. We demonstrate ultra-low-rate, low-latency, and channel-adaptive semantic communications.
• Score: 43.27015039765803
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative foundation AI models have recently shown great success in synthesizing natural signals with high perceptual quality using only textual prompts and conditioning signals to guide the generation process. This enables semantic communications at extremely low data rates in future wireless networks. In this paper, we develop a latency-aware semantic communications framework with pre-trained generative models. The transmitter performs multi-modal semantic decomposition on the input signal and transmits each semantic stream with the appropriate coding and communication schemes based on the intent. For the prompt, we adopt a re-transmission-based scheme to ensure reliable transmission, and for the other semantic modalities we use an adaptive modulation/coding scheme to achieve robustness to the changing wireless channel. Furthermore, we design a semantic and latency-aware scheme to allocate transmission power to different semantic modalities based on their importance subjected to semantic quality constraints. At the receiver, a pre-trained generative model synthesizes a high fidelity signal using the received multi-stream semantics. Simulation results demonstrate ultra-low-rate, low-latency, and channel-adaptive semantic communications.

Related papers

• Diffusion-based Generative Multicasting with Intent-aware Semantic Decomposition [45.73113352988143]
  We develop an intent-aware generative semantic multicasting framework utilizing pre-trained diffusion models. In the proposed framework, the transmitter decomposes the source signal to multiple semantic classes based on the multi-user intent. We show that the simulation results demonstrate significantly reduced per-user latency compared with non-generative and intent-unaware multicasting benchmarks.
  arXiv  Detail & Related papers  (2024-11-04T17:58:54Z)
• Generative Semantic Communication for Text-to-Speech Synthesis [39.8799066368712]
  This paper develops a novel generative semantic communication framework for text-to-speech synthesis. We employ a transformer encoder and a diffusion model to achieve efficient semantic coding without introducing significant communication overhead.
  arXiv  Detail & Related papers  (2024-10-04T14:18:31Z)
• Semantic Successive Refinement: A Generative AI-aided Semantic Communication Framework [27.524671767937512]
  We introduce a novel Generative AI Semantic Communication (GSC) system for single-user scenarios. At the transmitter end, it employs a joint source-channel coding mechanism based on the Swin Transformer for efficient semantic feature extraction. At the receiver end, an advanced Diffusion Model (DM) reconstructs high-quality images from degraded signals, enhancing perceptual details.
  arXiv  Detail & Related papers  (2024-07-31T06:08:51Z)
• Latent Diffusion Model-Enabled Real-Time Semantic Communication Considering Semantic Ambiguities and Channel Noises [18.539501941328393]
  This paper constructs a latent diffusion model-enabled SemCom system, and proposes three improvements compared to existing works. A lightweight single-layer latent space transformation adapter completes one-shot learning at the transmitter. An end-to-end consistency distillation strategy is used to distill the diffusion models trained in latent space.
  arXiv  Detail & Related papers  (2024-06-09T23:39:31Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Reasoning with the Theory of Mind for Pragmatic Semantic Communication [62.87895431431273]
  A pragmatic semantic communication framework is proposed in this paper. It enables effective goal-oriented information sharing between two-intelligent agents. Numerical evaluations demonstrate the framework's ability to achieve efficient communication with a reduced amount of bits.
  arXiv  Detail & Related papers  (2023-11-30T03:36:19Z)
• Generative AI-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts [89.04751776308656]
  This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding. In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
  arXiv  Detail & Related papers  (2023-09-05T23:24:56Z)
• Communication-Efficient Framework for Distributed Image Semantic Wireless Transmission [68.69108124451263]
  Federated learning-based semantic communication (FLSC) framework for multi-task distributed image transmission with IoT devices. Each link is composed of a hierarchical vision transformer (HVT)-based extractor and a task-adaptive translator. Channel state information-based multiple-input multiple-output transmission module designed to combat channel fading and noise.
  arXiv  Detail & Related papers  (2023-08-07T16:32:14Z)
• Communication Beyond Transmitting Bits: Semantics-Guided Source and Channel Coding [7.080957878208516]
  "Semantic communications" offers promising research direction. Injecting semantic guidance into the coded transmission design to achieve semantics-aware communications shows great potential for breakthrough in effectiveness and reliability. This article sheds light on semantics-guided source and channel coding as a transmission paradigm of semantic communications.
  arXiv  Detail & Related papers  (2022-08-04T06:12:55Z)
• Neuro-Symbolic Artificial Intelligence (AI) for Intent based Semantic Communication [85.06664206117088]
  6G networks must consider semantics and effectiveness (at end-user) of the data transmission. NeSy AI is proposed as a pillar for learning causal structure behind the observed data. GFlowNet is leveraged for the first time in a wireless system to learn the probabilistic structure which generates the data.
  arXiv  Detail & Related papers  (2022-05-22T07:11:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.