Token-Oriented Semantic Communication with Pretrained Vision Transformers

• URL: http://arxiv.org/abs/2608.25410v1
• Date: Wed, 26 Aug 2026 06:13:45 GMT
• Title: Token-Oriented Semantic Communication with Pretrained Vision Transformers
• Abstract summary: We propose a emphtoken-oriented semantic communication framework.<n>In this framework, token-level task relevance determines which compressed image latents are transmitted.<n>Experiments on ImageNet show that the proposed framework achieves a more favorable rate--accuracy trade-off than recent semantic communication schemes.
• Score: 19.377096772248418
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Token communications realize the semantic communication principle at the granularity of transformer tokens, providing a promising direction for client--server collaborative inference in resource-constrained edge systems. However, directly transmitting token embeddings presents two practical challenges: substantial communication cost and limited interoperability across model-specific token embedding spaces. To address these challenges, we propose a \emph{token-oriented} semantic communication framework. In this framework, token-level task relevance determines which compressed image latents are transmitted, enabling token-granular transmission without directly transmitting token embeddings. The framework is modular, coordinating three pretrained components---a lightweight client-side vision transformer (ViT), a learned image compression (LIC) model, and a large server-side ViT---without end-to-end training. The key enabler is the one-to-one spatial alignment between ViT patch tokens and the LIC latent vectors, which allows token-level task relevance to directly determine which latent vectors are transmitted. Building on this alignment, token-aligned LIC selectively transmits task-relevant latents, layer-selective attention rollout estimates token relevance from a selected range of attention layers in a single forward pass, and surrogate token substitution adapts the frozen server model by optimizing a single learnable token. Experiments on ImageNet show that the proposed framework achieves a more favorable rate--accuracy trade-off than recent semantic communication schemes, hand-crafted codecs, and task-agnostic LIC models.

Related papers

• Ada-TokenCom: Rate-Adaptive Token Communications via Large-Model-Driven Token Compression and Generation [52.99258064220833]
  Ada-TokenCom is a rate-adaptive TokenCom framework based on large autoregressive models.<n>We propose a mixed reconstruction/generation scheme, where the transmitter encodes and transmits the highly informative tokens.<n>We show that our proposed Ada-TokenCom framework outperforms both digital and deep joint source-channel coding-based semantic communication baselines.
  arXiv  Detail & Related papers  (2026-08-28T08:54:28Z)
• Explainable Task-Oriented Token Communication for AI-Native 6G Networks [66.32351608034986]
  The integration of Foundation Models (FMs) and wireless communications is driving the evolution of image communication from bit-accurate transmission toward task-oriented transmission.<n>Existing task-oriented image communication methods still face three major challenges: insufficient task-oriented Token representation, inadequate collaboration between Visual Tokens and Task Tokens, and limited interpretability of task decisions.<n>We propose an Explainable Task-Oriented Token Communication (ET-TokenCom) framework to address these challenges.
  arXiv  Detail & Related papers  (2026-06-12T05:52:08Z)
• Representative Attention For Vision Transformers [11.0298065364767]
  Linear attention has emerged as a promising direction for scaling Vision Transformers beyond the quadratic cost of dense self-attention.<n>We propose Representative Attention (RPAttention), a linear global attention mechanism that performs token compression directly in representation space.<n>RPAttention reduces the dominant token interaction complexity from quadratic to linear scaling with respect to the number of spatial tokens.
  arXiv  Detail & Related papers  (2026-05-14T14:48:20Z)
• Context-Aware Wireless Token Communication via Joint Token Masking and Detection [23.50554014275582]
  We propose a context-aware token communication framework that leverages a masked language model (MLM) as a shared contextual model between the transmitter (Tx) and receiver (Rx)<n>We develop a context-aware token detection method that integrates channel likelihoods with contextual priors under a Bayesian formulation, enabling robust token inference over noisy channels.<n>We show that the proposed framework significantly improves reconstruction performance compared to conventional and existing token communication schemes.
  arXiv  Detail & Related papers  (2026-05-04T01:06:04Z)
• Wireless TokenCom: RL-Based Tokenizer Agreement for Multi-User Wireless Token Communications [59.84545048095092]
  Token Communications (TokenCom) has recently emerged as an effective new paradigm, where tokens are the unified units of communications computations.<n>We investigate a multi-user downlink wireless TokenCom scenario, where the base station transmits multiple users.
  arXiv  Detail & Related papers  (2026-02-12T19:00:33Z)
• SC-GIR: Goal-oriented Semantic Communication via Invariant Representation Learning [59.45312293893698]
  Goal-oriented semantic communication (SC) aims to revolutionize communication systems by transmitting only task-essential information.<n>We propose a novel framework called Goal-oriented Invariant Representation-based SC (SC-GIR) for image transmission.
  arXiv  Detail & Related papers  (2025-09-01T04:29:43Z)
• Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach [55.861432910722186]
  UniToCom is a unified token communication paradigm that treats tokens as the fundamental units for both processing and wireless transmission.<n>We propose a generative information bottleneck (GenIB) principle, which facilitates the learning of tokens that preserve essential information.<n>We employ a causal Transformer-based multimodal large language model (MLLM) at the receiver to unify the processing of both discrete and continuous tokens.
  arXiv  Detail & Related papers  (2025-07-02T14:03:01Z)
• Adaptive Semantic Token Communication for Transformer-based Edge Inference [15.405730528104113]
  This paper presents an adaptive framework for edge inference based on a dynamically transformer-powered deep joint source channel coding architecture.<n>We employ a semantic token selection mechanism that adaptively compresses informative features into a user specified number of tokens per sample.<n>We incorporate a resource allocation algorithm based on Lyapunov optimization to enhance robustness under dynamic network conditions.
  arXiv  Detail & Related papers  (2025-05-23T08:15:05Z)
• Token Communications: A Large Model-Driven Framework for Cross-modal Context-aware Semantic Communications [78.80966346820553]
  We introduce token communications (TokCom), a large model-driven framework to leverage cross-modal context information in generative semantic communications (GenSC)<n>In this paper, we introduce the potential opportunities and challenges of leveraging context in GenSC, explore how to integrate GFM/MLLMs-based token processing into semantic communication systems, present the key principles for efficient TokCom at various layers in future wireless networks.
  arXiv  Detail & Related papers  (2025-02-17T18:14:18Z)
• Adaptive Semantic Token Selection for AI-native Goal-oriented Communications [11.92172357956248]
  We propose a novel design for AI-native goal-oriented communications. We exploit transformer neural networks under dynamic inference constraints on bandwidth and computation. We show that our model improves over state-of-the-art token selection mechanisms.
  arXiv  Detail & Related papers  (2024-04-25T13:49:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.