Agent-driven Generative Semantic Communication with Cross-Modality and Prediction
- URL: http://arxiv.org/abs/2404.06997v2
- Date: Fri, 19 Jul 2024 09:28:54 GMT
- Title: Agent-driven Generative Semantic Communication with Cross-Modality and Prediction
- Authors: Wanting Yang, Zehui Xiong, Yanli Yuan, Wenchao Jiang, Tony Q. S. Quek, Merouane Debbah,
- Abstract summary: We propose a novel agent-driven generative semantic communication framework based on reinforcement learning.
In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling.
The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
- Score: 57.335922373309074
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In the era of 6G, with compelling visions of intelligent transportation systems and digital twins, remote surveillance is poised to become a ubiquitous practice. Substantial data volume and frequent updates present challenges in wireless networks. To address these challenges, we propose a novel agent-driven generative semantic communication (A-GSC) framework based on reinforcement learning. In contrast to the existing research on semantic communication (SemCom), which mainly focuses on either semantic extraction or semantic sampling, we seamlessly integrate both by jointly considering the intrinsic attributes of source information and the contextual information regarding the task. Notably, the introduction of generative artificial intelligence (GAI) enables the independent design of semantic encoders and decoders. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. Accordingly, we design a semantic decoder with both predictive and generative capabilities, consisting of two tailored modules. Moreover, the effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework in both energy saving and reconstruction accuracy.
Related papers
- Generative AI-aided Joint Training-free Secure Semantic Communications
via Multi-modal Prompts [89.04751776308656]
This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding.
In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
arXiv Detail & Related papers (2023-09-05T23:24:56Z) - Enhancing Semantic Communication with Deep Generative Models -- An
ICASSP Special Session Overview [25.314693624878053]
This ICASSP special session overview paper discloses the semantic communication challenges from the machine learning perspective.
It unveils how deep generative models will significantly enhance semantic communication frameworks.
It charts novel research pathways for the next generative semantic communication frameworks.
arXiv Detail & Related papers (2023-09-05T15:11:16Z) - Causal Semantic Communication for Digital Twins: A Generalizable
Imitation Learning Approach [74.25870052841226]
A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing, and artificial intelligence (AI) technologies to enable many connected intelligence services.
Wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints.
A novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems.
arXiv Detail & Related papers (2023-04-25T00:15:00Z) - Enabling the Wireless Metaverse via Semantic Multiverse Communication [82.47169682083806]
Metaverse over wireless networks is an emerging use case of the sixth generation (6G) wireless systems.
We propose a novel semantic communication framework by decomposing the metaverse into human/machine agent-specific semantic multiverses (SMs)
An SM stored at each agent comprises a semantic encoder and a generator, leveraging recent advances in generative artificial intelligence (AI)
arXiv Detail & Related papers (2022-12-13T21:21:07Z) - Seq2Seq-SC: End-to-End Semantic Communication Systems with Pre-trained
Language Model [20.925910474226885]
We propose a realistic semantic network called seq2seq-SC, designed to be compatible with 5G NR.
We employ a performance metric called semantic similarity, measured by BLEU for lexical similarity and SBERT for semantic similarity.
arXiv Detail & Related papers (2022-10-27T07:48:18Z) - Task-Oriented Sensing, Computation, and Communication Integration for
Multi-Device Edge AI [108.08079323459822]
This paper studies a new multi-intelligent edge artificial-latency (AI) system, which jointly exploits the AI model split inference and integrated sensing and communication (ISAC)
We measure the inference accuracy by adopting an approximate but tractable metric, namely discriminant gain.
arXiv Detail & Related papers (2022-07-03T06:57:07Z) - Neuro-Symbolic Artificial Intelligence (AI) for Intent based Semantic
Communication [85.06664206117088]
6G networks must consider semantics and effectiveness (at end-user) of the data transmission.
NeSy AI is proposed as a pillar for learning causal structure behind the observed data.
GFlowNet is leveraged for the first time in a wireless system to learn the probabilistic structure which generates the data.
arXiv Detail & Related papers (2022-05-22T07:11:57Z) - Wireless Resource Management in Intelligent Semantic Communication
Networks [15.613654766345702]
We address the user association (UA) and bandwidth allocation problems in an ISC-enabled heterogeneous network (ISC-HetNet)
We propose a two-stage solution, including a programming method to obtain an objective, and a algorithm in the second stage to reach the optimality of UA and BA.
arXiv Detail & Related papers (2022-02-15T18:28:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.