Token Communications: A Unified Framework for Cross-modal Context-aware Semantic Communications

• URL: http://arxiv.org/abs/2502.12096v2
• Date: Fri, 06 Jun 2025 18:53:50 GMT
• Title: Token Communications: A Unified Framework for Cross-modal Context-aware Semantic Communications
• Authors: Li Qiao, Mahdi Boloursaz Mashhadi, Zhen Gao, Rahim Tafazolli, Mehdi Bennis, Dusit Niyato,
• Abstract summary: We introduce token communications (TokCom), a large model-driven framework to leverage cross-modal context information in generative semantic communications (GenSC)<n>In this paper, we introduce the potential opportunities and challenges of leveraging context in GenSC, explore how to integrate GFM/MLLMs-based token processing into semantic communication systems, present the key principles for efficient TokCom at various layers in future wireless networks.
• Score: 78.80966346820553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we introduce token communications (TokCom), a large model-driven framework to leverage cross-modal context information in generative semantic communications (GenSC). TokCom is a new paradigm, motivated by the recent success of generative foundation models and multimodal large language models (GFM/MLLMs), where the communication units are tokens, enabling efficient transformer-based token processing at the transmitter and receiver. In this paper, we introduce the potential opportunities and challenges of leveraging context in GenSC, explore how to integrate GFM/MLLMs-based token processing into semantic communication systems to leverage cross-modal context effectively at affordable complexity, present the key principles for efficient TokCom at various layers in future wireless networks. In a typical image semantic communication setup, we demonstrate a significant improvement of the bandwidth efficiency, achieved by TokCom by leveraging the context information among tokens. Finally, the potential research directions are identified to facilitate adoption of TokCom in future wireless networks.

Related papers

• Engineered over Emergent Communication in MARL for Scalable and Sample-Efficient Cooperative Task Allocation in a Partially Observable Grid [0.0]
  We compare the efficacy of learned versus engineered communication strategies in a cooperative multi-agent reinforcement learning (MARL) environment.<n>For the learned approach, we introduce Learned Direct Communication (LDC), where agents generate messages and actions concurrently via a neural network.<n>Our engineered approach, Intention Communication, employs an Imagined Trajectory Generation Module (ITGM) and a Message Generation Network (MGN) to formulate messages based on predicted future states.
  arXiv  Detail & Related papers  (2025-08-04T21:29:07Z)
• Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach [55.861432910722186]
  UniToCom is a unified token communication paradigm that treats tokens as the fundamental units for both processing and wireless transmission.<n>We propose a generative information bottleneck (GenIB) principle, which facilitates the learning of tokens that preserve essential information.<n>We employ a causal Transformer-based multimodal large language model (MLLM) at the receiver to unify the processing of both discrete and continuous tokens.
  arXiv  Detail & Related papers  (2025-07-02T14:03:01Z)
• Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning [9.48183472865413]
  We develop a scalable communication protocol for cooperative multi-agent reinforcement learning (MARL)<n>We propose utilizing the exponential topology to enable rapid information dissemination among agents by leveraging its small-diameter and small-size properties.<n>Experiments on large-scale cooperative benchmarks, including MAgent and Infrastructure Management Planning, demonstrate the superior performance and robust zero-shot transferability of ExpoComm.
  arXiv  Detail & Related papers  (2025-02-27T03:15:31Z)
• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Semantic Revolution from Communications to Orchestration for 6G: Challenges, Enablers, and Research Directions [16.807697160355303]
  This paper introduces the Knowledge Base Management And Orchestration (KB-MANO) framework. KB-MANO is crafted for the allocation of network and computing resources dedicated to updating and redistributing knowledge. A proof-of-concept is proposed to showcase the integration of KB-MANO with resource allocation in radio access networks.
  arXiv  Detail & Related papers  (2024-06-24T09:04:09Z)
• Will 6G be Semantic Communications? Opportunities and Challenges from Task Oriented and Secure Communications to Integrated Sensing [49.83882366499547]
  This paper explores opportunities and challenges of task (goal)-oriented and semantic communications for next-generation (NextG) networks through the integration of multi-task learning. We employ deep neural networks representing a dedicated encoder at the transmitter and multiple task-specific decoders at the receiver. We scrutinize potential vulnerabilities stemming from adversarial attacks during both training and testing phases.
  arXiv  Detail & Related papers  (2024-01-03T04:01:20Z)
• Generative AI-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts [89.04751776308656]
  This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding. In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
  arXiv  Detail & Related papers  (2023-09-05T23:24:56Z)
• Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications [55.65768284748698]
  Machine learning (ML) is a promising enabler for the fifth generation (5G) communication systems and beyond. This article aims to provide a holistic overview of relevant communication and ML principles, and thereby present communication-efficient and distributed learning frameworks with selected use cases.
  arXiv  Detail & Related papers  (2020-08-06T12:37:14Z)
• Learning Structured Communication for Multi-agent Reinforcement Learning [104.64584573546524]
  This work explores the large-scale multi-agent communication mechanism under a multi-agent reinforcement learning (MARL) setting. We propose a novel framework termed as Learning Structured Communication (LSC) by using a more flexible and efficient communication topology.
  arXiv  Detail & Related papers  (2020-02-11T07:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.