Generative Semantic Communication: Architectures, Technologies, and Applications

• URL: http://arxiv.org/abs/2412.08642v1
• Date: Wed, 11 Dec 2024 18:59:50 GMT
• Title: Generative Semantic Communication: Architectures, Technologies, and Applications
• Authors: Jinke Ren, Yaping Sun, Hongyang Du, Weiwen Yuan, Chongjie Wang, Xianda Wang, Yingbin Zhou, Ziwei Zhu, Fangxin Wang, Shuguang Cui,
• Abstract summary: This paper delves into the applications of generative artificial intelligence (GAI) in semantic communication (SemCom) Three popular SemCom systems are first introduced, including variational autoencoders, generative adversarial networks, and diffusion models. A novel generative SemCom system is proposed by incorporating the cutting-edge GAI technology-large language models (LLMs)
• Score: 36.67865904029129
• License:
• Abstract: This paper delves into the applications of generative artificial intelligence (GAI) in semantic communication (SemCom) and presents a thorough study. Three popular SemCom systems enabled by classical GAI models are first introduced, including variational autoencoders, generative adversarial networks, and diffusion models. For each system, the fundamental concept of the GAI model, the corresponding SemCom architecture, and the associated literature review of recent efforts are elucidated. Then, a novel generative SemCom system is proposed by incorporating the cutting-edge GAI technology-large language models (LLMs). This system features two LLM-based AI agents at both the transmitter and receiver, serving as "brains" to enable powerful information understanding and content regeneration capabilities, respectively. This innovative design allows the receiver to directly generate the desired content, instead of recovering the bit stream, based on the coded semantic information conveyed by the transmitter. Therefore, it shifts the communication mindset from "information recovery" to "information regeneration" and thus ushers in a new era of generative SemCom. A case study on point-to-point video retrieval is presented to demonstrate the superiority of the proposed generative SemCom system, showcasing a 99.98% reduction in communication overhead and a 53% improvement in retrieval accuracy compared to the traditional communication system. Furthermore, four typical application scenarios for generative SemCom are delineated, followed by a discussion of three open issues warranting future investigation. In a nutshell, this paper provides a holistic set of guidelines for applying GAI in SemCom, paving the way for the efficient implementation of generative SemCom in future wireless networks.

Related papers

• Large Generative Model-assisted Talking-face Semantic Communication System [55.42631520122753]
  This study introduces a Large Generative Model-assisted Talking-face Semantic Communication (LGM-TSC) system. Generative Semantic Extractor (GSE) at the transmitter converts semantically sparse talking-face videos into texts with high information density. Private Knowledge Base (KB) based on the Large Language Model (LLM) for semantic disambiguation and correction. Generative Semantic Reconstructor (GSR) that utilizes BERT-VITS2 and SadTalker models to transform text back into a high-QoE talking-face video.
  arXiv  Detail & Related papers  (2024-11-06T12:45:46Z)
• Rethinking Multi-User Semantic Communications with Deep Generative Models [30.745379375963157]
  We develop a novel generative semantic communication framework tailored for multi-user scenarios. Under this innovative perspective, OFDMA systems should not aim to transmit the largest part of information. The thorough experimental evaluation shows the capabilities of the novel diffusion model and the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2024-05-16T07:43:15Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• A Wireless AI-Generated Content (AIGC) Provisioning Framework Empowered by Semantic Communication [53.78269720999609]
  This paper proposes a semantic communication (SemCom)-empowered AIGC (SemAIGC) generation and transmission framework. Specifically, SemAIGC integrates diffusion models within the semantic encoder and decoder to design a workload-adjustable transceiver. Simulations verify the superiority of our proposed SemAIGC framework in terms of latency and content quality compared to conventional approaches.
  arXiv  Detail & Related papers  (2023-10-26T18:05:22Z)
• Generative AI-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts [89.04751776308656]
  This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding. In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
  arXiv  Detail & Related papers  (2023-09-05T23:24:56Z)
• Enhancing Semantic Communication with Deep Generative Models -- An ICASSP Special Session Overview [25.314693624878053]
  This ICASSP special session overview paper discloses the semantic communication challenges from the machine learning perspective. It unveils how deep generative models will significantly enhance semantic communication frameworks. It charts novel research pathways for the next generative semantic communication frameworks.
  arXiv  Detail & Related papers  (2023-09-05T15:11:16Z)
• Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach [74.25870052841226]
  A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing, and artificial intelligence (AI) technologies to enable many connected intelligence services. Wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints. A novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems.
  arXiv  Detail & Related papers  (2023-04-25T00:15:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.