Vision Transformer Based Semantic Communications for Next Generation Wireless Networks

• URL: http://arxiv.org/abs/2503.17275v1
• Date: Fri, 21 Mar 2025 16:23:02 GMT
• Title: Vision Transformer Based Semantic Communications for Next Generation Wireless Networks
• Authors: Muhammad Ahmed Mohsin, Muhammad Jazib, Zeeshan Alam, Muhmmad Farhan Khan, Muhammad Saad, Muhammad Ali Jamshed,
• Abstract summary: This paper presents a Vision Transformer (ViT)-based semantic communication framework.<n>By equipping ViT as the encoder-decoder framework, the proposed architecture can proficiently encode images into a high semantic content.<n>The architecture based on the proposed ViT network achieves the Peak Signal-versato-noise Ratio (PSNR) of 38 dB.
• Score: 3.8095664680229935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the evolving landscape of 6G networks, semantic communications are poised to revolutionize data transmission by prioritizing the transmission of semantic meaning over raw data accuracy. This paper presents a Vision Transformer (ViT)-based semantic communication framework that has been deliberately designed to achieve high semantic similarity during image transmission while simultaneously minimizing the demand for bandwidth. By equipping ViT as the encoder-decoder framework, the proposed architecture can proficiently encode images into a high semantic content at the transmitter and precisely reconstruct the images, considering real-world fading and noise consideration at the receiver. Building on the attention mechanisms inherent to ViTs, our model outperforms Convolution Neural Network (CNNs) and Generative Adversarial Networks (GANs) tailored for generating such images. The architecture based on the proposed ViT network achieves the Peak Signal-to-noise Ratio (PSNR) of 38 dB, which is higher than other Deep Learning (DL) approaches in maintaining semantic similarity across different communication environments. These findings establish our ViT-based approach as a significant breakthrough in semantic communications.

Related papers

• Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model [55.71885688565501]
  We propose a scalable generative video semantic communication framework that extracts and transmits semantic information to achieve high-quality video reconstruction.<n>Specifically, at the transmitter, description and other condition signals are extracted from the source video, functioning as text and structural semantics, respectively.<n>At the receiver, the diffusion-based GenAI large models are utilized to fuse the semantics of the multiple modalities for reconstructing the video.
  arXiv  Detail & Related papers  (2025-02-19T15:59:07Z)
• Semantic Communication based on Generative AI: A New Approach to Image Compression and Edge Optimization [1.450405446885067]
  This thesis integrates semantic communication and generative models for optimized image compression and edge network resource allocation.<n>The communication infrastructure can benefit to significant improvements in bandwidth efficiency and latency reduction.<n>Results demonstrate the potential of combining generative AI and semantic communication to create more efficient semantic-goal-oriented communication networks.
  arXiv  Detail & Related papers  (2025-02-01T21:48:31Z)
• Semantic Feature Decomposition based Semantic Communication System of Images with Large-scale Visual Generation Models [5.867765921443141]
  A Texture-Color based Semantic Communication system of Images TCSCI is proposed. It decomposing the images into their natural language description (text), texture and color semantic features at the transmitter. It can achieve extremely compressed, highly noise-resistant, and visually similar image semantic communication, while ensuring the interpretability and editability of the transmission process.
  arXiv  Detail & Related papers  (2024-10-26T08:53:05Z)
• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Federated Multi-View Synthesizing for Metaverse [52.59476179535153]
  The metaverse is expected to provide immersive entertainment, education, and business applications. Virtual reality (VR) transmission over wireless networks is data- and computation-intensive. We have developed a novel multi-view synthesizing framework that can efficiently provide synthesizing, storage, and communication resources for wireless content delivery in the metaverse.
  arXiv  Detail & Related papers  (2023-12-18T13:51:56Z)
• Communication-Efficient Framework for Distributed Image Semantic Wireless Transmission [68.69108124451263]
  Federated learning-based semantic communication (FLSC) framework for multi-task distributed image transmission with IoT devices. Each link is composed of a hierarchical vision transformer (HVT)-based extractor and a task-adaptive translator. Channel state information-based multiple-input multiple-output transmission module designed to combat channel fading and noise.
  arXiv  Detail & Related papers  (2023-08-07T16:32:14Z)
• Wireless End-to-End Image Transmission System using Semantic Communications [4.2421412410466575]
  The research shows that the resource gain in the form of bandwidth saving is immense when transmitting the semantic segmentation map through the physical channel. The research studies the effect of physical channel distortions and quantization noise on semantic communication-based multimedia content transmission.
  arXiv  Detail & Related papers  (2023-02-27T12:33:53Z)
• Enabling the Wireless Metaverse via Semantic Multiverse Communication [82.47169682083806]
  Metaverse over wireless networks is an emerging use case of the sixth generation (6G) wireless systems. We propose a novel semantic communication framework by decomposing the metaverse into human/machine agent-specific semantic multiverses (SMs) An SM stored at each agent comprises a semantic encoder and a generator, leveraging recent advances in generative artificial intelligence (AI)
  arXiv  Detail & Related papers  (2022-12-13T21:21:07Z)
• Demo: Real-Time Semantic Communications with a Vision Transformer [14.85519988496995]
  We propose an end-to-end deep neural network-based architecture for image transmission and demonstrate its feasibility in a real-time wireless channel. To the best of our knowledge, this is the first work that implements and investigates real-time semantic communications with a vision transformer.
  arXiv  Detail & Related papers  (2022-05-08T14:49:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.