Token Communications: A Unified Framework for Cross-modal Context-aware Semantic Communications

• URL: http://arxiv.org/abs/2502.12096v1
• Date: Mon, 17 Feb 2025 18:14:18 GMT
• Title: Token Communications: A Unified Framework for Cross-modal Context-aware Semantic Communications
• Authors: Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Rahim Tafazolli, Mehdi Bennis, Dusit Niyato,
• Abstract summary: We introduce token communications (TokCom), a unified framework to leverage cross-modal context information in generative semantic communications (GenSC) TokCom is motivated by the recent success of generative foundation models and multimodal large language models (GFM/MLLMs) We demonstrate the corresponding TokCom benefits in a GenSC setup for image, leveraging cross-modal context information, which increases the bandwidth efficiency by 70.8% with negligible loss of semantic/perceptual quality.
• Score: 78.80966346820553
• License:
• Abstract: In this paper, we introduce token communications (TokCom), a unified framework to leverage cross-modal context information in generative semantic communications (GenSC). TokCom is a new paradigm, motivated by the recent success of generative foundation models and multimodal large language models (GFM/MLLMs), where the communication units are tokens, enabling efficient transformer-based token processing at the transmitter and receiver. In this paper, we introduce the potential opportunities and challenges of leveraging context in GenSC, explore how to integrate GFM/MLLMs-based token processing into semantic communication systems to leverage cross-modal context effectively, present the key principles for efficient TokCom at various layers in future wireless networks. We demonstrate the corresponding TokCom benefits in a GenSC setup for image, leveraging cross-modal context information, which increases the bandwidth efficiency by 70.8% with negligible loss of semantic/perceptual quality. Finally, the potential research directions are identified to facilitate adoption of TokCom in future wireless networks.

Related papers

• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Semantic Revolution from Communications to Orchestration for 6G: Challenges, Enablers, and Research Directions [16.807697160355303]
  This paper introduces the Knowledge Base Management And Orchestration (KB-MANO) framework. KB-MANO is crafted for the allocation of network and computing resources dedicated to updating and redistributing knowledge. A proof-of-concept is proposed to showcase the integration of KB-MANO with resource allocation in radio access networks.
  arXiv  Detail & Related papers  (2024-06-24T09:04:09Z)
• Will 6G be Semantic Communications? Opportunities and Challenges from Task Oriented and Secure Communications to Integrated Sensing [49.83882366499547]
  This paper explores opportunities and challenges of task (goal)-oriented and semantic communications for next-generation (NextG) networks through the integration of multi-task learning. We employ deep neural networks representing a dedicated encoder at the transmitter and multiple task-specific decoders at the receiver. We scrutinize potential vulnerabilities stemming from adversarial attacks during both training and testing phases.
  arXiv  Detail & Related papers  (2024-01-03T04:01:20Z)
• Generative AI-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts [89.04751776308656]
  This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding. In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
  arXiv  Detail & Related papers  (2023-09-05T23:24:56Z)
• Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications [55.65768284748698]
  Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. This article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases.
  arXiv  Detail & Related papers  (2020-08-06T12:37:14Z)
• Learning Structured Communication for Multi-agent Reinforcement Learning [104.64584573546524]
  This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology.
  arXiv  Detail & Related papers  (2020-02-11T07:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.