Adaptive Semantic Token Communication for Transformer-based Edge Inference

• URL: http://arxiv.org/abs/2505.17604v1
• Date: Fri, 23 May 2025 08:15:05 GMT
• Title: Adaptive Semantic Token Communication for Transformer-based Edge Inference
• Authors: Alessio Devoto, Jary Pomponi, Mattia Merluzzi, Paolo Di Lorenzo, Simone Scardapane,
• Abstract summary: This paper presents an adaptive framework for edge inference based on a dynamically transformer-powered deep joint source channel coding architecture.<n>We employ a semantic token selection mechanism that adaptively compresses informative features into a user specified number of tokens per sample.<n>We incorporate a resource allocation algorithm based on Lyapunov optimization to enhance robustness under dynamic network conditions.
• Score: 15.405730528104113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents an adaptive framework for edge inference based on a dynamically configurable transformer-powered deep joint source channel coding (DJSCC) architecture. Motivated by a practical scenario where a resource constrained edge device engages in goal oriented semantic communication, such as selectively transmitting essential features for object detection to an edge server, our approach enables efficient task aware data transmission under varying bandwidth and channel conditions. To achieve this, input data is tokenized into compact high level semantic representations, refined by a transformer, and transmitted over noisy wireless channels. As part of the DJSCC pipeline, we employ a semantic token selection mechanism that adaptively compresses informative features into a user specified number of tokens per sample. These tokens are then further compressed through the JSCC module, enabling a flexible token communication strategy that adjusts both the number of transmitted tokens and their embedding dimensions. We incorporate a resource allocation algorithm based on Lyapunov stochastic optimization to enhance robustness under dynamic network conditions, effectively balancing compression efficiency and task performance. Experimental results demonstrate that our system consistently outperforms existing baselines, highlighting its potential as a strong foundation for AI native semantic communication in edge intelligence applications.

Related papers

• VQ-DeepISC: Vector Quantized-Enabled Digital Semantic Communication with Channel Adaptive Image Transmission [8.858565507331395]
  Discretization of semantic features enables interoperability between semantic and digital communication systems.<n>We propose a vector quantized (VQ)-enabled digital semantic communication system with channel adaptive image transmission.
  arXiv  Detail & Related papers  (2025-08-01T02:35:34Z)
• Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach [55.861432910722186]
  UniToCom is a unified token communication paradigm that treats tokens as the fundamental units for both processing and wireless transmission.<n>We propose a generative information bottleneck (GenIB) principle, which facilitates the learning of tokens that preserve essential information.<n>We employ a causal Transformer-based multimodal large language model (MLLM) at the receiver to unify the processing of both discrete and continuous tokens.
  arXiv  Detail & Related papers  (2025-07-02T14:03:01Z)
• Task-Oriented Low-Label Semantic Communication With Self-Supervised Learning [67.06363342414397]
  Task-oriented semantic communication enhances transmission efficiency by conveying semantic information rather than exact messages.<n>Deep learning (DL)-based semantic communication can effectively cultivate the essential semantic knowledge for semantic extraction, transmission, and interpretation.<n>We propose a self-supervised learning-based semantic communication framework (SLSCom) to enhance task inference performance.
  arXiv  Detail & Related papers  (2025-05-26T13:06:18Z)
• Modeling and Performance Analysis for Semantic Communications Based on Empirical Results [53.805458017074294]
  We propose an Alpha-Beta-Gamma (ABG) formula to model the relationship between the end-to-end measurement and SNR.<n>For image reconstruction tasks, the proposed ABG formula can well fit the commonly used DL networks, such as SCUNet, and Vision Transformer.<n>To the best of our knowledge, this is the first theoretical expression between end-to-end performance metrics and SNR for semantic communications.
  arXiv  Detail & Related papers  (2025-04-29T06:07:50Z)
• Take What You Need: Flexible Multi-Task Semantic Communications with Channel Adaptation [51.53221300103261]
  This article introduces a novel channel-adaptive and multi-task-aware semantic communication framework based on a masked auto-encoder architecture.<n>A channel-aware extractor is employed to dynamically select relevant information in response to real-time channel conditions.<n> Experimental results demonstrate the superior performance of our framework compared to conventional methods in tasks such as image reconstruction and object detection.
  arXiv  Detail & Related papers  (2025-02-12T09:01:25Z)
• Efficient Semantic Communication Through Transformer-Aided Compression [31.285983939625098]
  We introduce a channel-aware adaptive framework for semantic communication.<n>By employing vision transformers, we interpret the attention mask as a measure of the semantic contents of the patches.<n>Our method enhances communication efficiency by adapting the encoding resolution to the content's relevance.
  arXiv  Detail & Related papers  (2024-12-02T18:57:28Z)
• Adaptive Semantic Token Selection for AI-native Goal-oriented Communications [11.92172357956248]
  We propose a novel design for AI-native goal-oriented communications. We exploit transformer neural networks under dynamic inference constraints on bandwidth and computation. We show that our model improves over state-of-the-art token selection mechanisms.
  arXiv  Detail & Related papers  (2024-04-25T13:49:50Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Adaptive Resource Allocation for Semantic Communication Networks [34.189531352110386]
  This paper investigates the quality of service for semantic communication networks, including the semantic quantization efficiency (SQE) and transmission latency. A problem maximizing the overall effective SC-QoS is formulated by jointly the transmit beamforming the base station, the bits semantic representation the subchannel assignment, and the semantic resource allocation. Our design can effectively combat semantic noise and achieve superior performance in wireless communications compared to several benchmark schemes.
  arXiv  Detail & Related papers  (2023-12-02T09:12:12Z)
• Task-Oriented Sensing, Computation, and Communication Integration for Multi-Device Edge AI [108.08079323459822]
  This paper studies a new multi-intelligent edge artificial-latency (AI) system, which jointly exploits the AI model split inference and integrated sensing and communication (ISAC) We measure the inference accuracy by adopting an approximate but tractable metric, namely discriminant gain.
  arXiv  Detail & Related papers  (2022-07-03T06:57:07Z)
• Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach [3.983055670167878]
  A low-end edge device transmits the extracted feature vector of a local data sample to a powerful edge server for processing. It is critical to encode the data into an informative and compact representation for low-latency inference given the limited bandwidth. We propose a learning-based communication scheme that jointly optimize feature extraction, source coding, and channel coding.
  arXiv  Detail & Related papers  (2021-02-08T12:53:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.