Semantic Communications with Computer Vision Sensing for Edge Video Transmission

• URL: http://arxiv.org/abs/2503.07252v1
• Date: Mon, 10 Mar 2025 12:34:22 GMT
• Title: Semantic Communications with Computer Vision Sensing for Edge Video Transmission
• Authors: Yubo Peng, Luping Xiang, Kun Yang, Kezhi Wang, Merouane Debbah,
• Abstract summary: Semantic communication (SC) offers a solution by extracting and compressing information at the semantic level.<n>Traditional SC methods face inefficiencies due to the repeated transmission of static frames in edge videos.<n>We propose a SC with computer vision sensing framework for edge video transmission.
• Score: 16.56792633171318
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the widespread adoption of vision sensors in edge applications, such as surveillance, the transmission of video data consumes substantial spectrum resources. Semantic communication (SC) offers a solution by extracting and compressing information at the semantic level, preserving the accuracy and relevance of transmitted data while significantly reducing the volume of transmitted information. However, traditional SC methods face inefficiencies due to the repeated transmission of static frames in edge videos, exacerbated by the absence of sensing capabilities, which results in spectrum inefficiency. To address this challenge, we propose a SC with computer vision sensing (SCCVS) framework for edge video transmission. The framework first introduces a compression ratio (CR) adaptive SC (CRSC) model, capable of adjusting CR based on whether the frames are static or dynamic, effectively conserving spectrum resources. Additionally, we implement an object detection and semantic segmentation models-enabled sensing (OSMS) scheme, which intelligently senses the changes in the scene and assesses the significance of each frame through in-context analysis. Hence, The OSMS scheme provides CR prompts to the CRSC model based on real-time sensing results. Moreover, both CRSC and OSMS are designed as lightweight models, ensuring compatibility with resource-constrained sensors commonly used in practical edge applications. Experimental simulations validate the effectiveness of the proposed SCCVS framework, demonstrating its ability to enhance transmission efficiency without sacrificing critical semantic information.

Related papers

• Optimal Transport Adapter Tuning for Bridging Modality Gaps in Few-Shot Remote Sensing Scene Classification [80.83325513157637]
  Few-Shot Remote Sensing Scene Classification (FS-RSSC) presents the challenge of classifying remote sensing images with limited labeled samples. We propose a novel Optimal Transport Adapter Tuning (OTAT) framework aimed at constructing an ideal Platonic representational space.
  arXiv  Detail & Related papers  (2025-03-19T07:04:24Z)
• Efficient Semantic Communication Through Transformer-Aided Compression [31.285983939625098]
  We introduce a channel-aware adaptive framework for semantic communication.<n>By employing vision transformers, we interpret the attention mask as a measure of the semantic contents of the patches.<n>Our method enhances communication efficiency by adapting the encoding resolution to the content's relevance.
  arXiv  Detail & Related papers  (2024-12-02T18:57:28Z)
• Cross-Layer Encrypted Semantic Communication Framework for Panoramic Video Transmission [11.438045765196332]
  We propose a cross-layer encrypted semantic communication (CLESC) framework for panoramic video transmission. We propose an adaptive cross-layer transmission mechanism that dynamically adjusts CRC, channel coding, and retransmission schemes based on the importance of semantic information. Compared to traditional cross-layer transmission schemes, the CLESC framework can reduce bandwidth consumption by 85%.
  arXiv  Detail & Related papers  (2024-11-19T07:18:38Z)
• Conformal Distributed Remote Inference in Sensor Networks Under Reliability and Communication Constraints [61.62410595953275]
  Communication-constrained distributed conformal risk control (CD-CRC)<n>CD-CRC is a novel decision-making framework for sensor networks under communication constraints.
  arXiv  Detail & Related papers  (2024-09-12T10:12:43Z)
• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Object-Attribute-Relation Representation Based Video Semantic Communication [35.87160453583808]
  We introduce the use of object-attribute-relation (OAR) as a semantic framework for videos to facilitate low bit-rate coding.<n>We utilize OAR sequences for both low bit-rate representation and generative video reconstruction.<n>Our experiments on traffic surveillance video datasets assess the effectiveness of our approach in terms of video transmission performance.
  arXiv  Detail & Related papers  (2024-06-15T02:19:31Z)
• Visual Language Model based Cross-modal Semantic Communication Systems [42.321208020228894]
  We propose a novel Vision-Language Model-based Cross-modal Semantic Communication system. The VLM-CSC comprises three novel components. The experimental simulations validate the effectiveness, adaptability, and robustness of the CSC system.
  arXiv  Detail & Related papers  (2024-05-06T08:59:16Z)
• Agent-driven Generative Semantic Communication with Cross-Modality and Prediction [57.335922373309074]
  We propose a novel agent-driven generative semantic communication framework based on reinforcement learning. In this work, we develop an agent-assisted semantic encoder with cross-modality capability, which can track the semantic changes, channel condition, to perform adaptive semantic extraction and sampling. The effectiveness of the designed models has been verified using the UA-DETRAC dataset, demonstrating the performance gains of the overall A-GSC framework.
  arXiv  Detail & Related papers  (2024-04-10T13:24:27Z)
• Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach [74.25870052841226]
  A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing, and artificial intelligence (AI) technologies to enable many connected intelligence services. Wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints. A novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems.
  arXiv  Detail & Related papers  (2023-04-25T00:15:00Z)
• Semantic Communication Enabling Robust Edge Intelligence for Time-Critical IoT Applications [87.05763097471487]
  This paper aims to design robust Edge Intelligence using semantic communication for time-critical IoT applications. We analyze the effect of image DCT coefficients on inference accuracy and propose the channel-agnostic effectiveness encoding for offloading.
  arXiv  Detail & Related papers  (2022-11-24T20:13:17Z)
• Robust Information Bottleneck for Task-Oriented Communication with Digital Modulation [31.39386509261528]
  Task-oriented communications, mostly using learning-based joint source-channel coding (JSCC), aim to design a communication-efficient edge inference system. We develop a robust encoding framework, named robust information bottleneck (RIB), to improve the communication robustness to the channel variations. The proposed DT-JSCC achieves better inference performance than the baseline methods with low communication latency.
  arXiv  Detail & Related papers  (2022-09-21T14:21:14Z)
• Model-based Deep Learning Receiver Design for Rate-Splitting Multiple Access [65.21117658030235]
  This work proposes a novel design for a practical RSMA receiver based on model-based deep learning (MBDL) methods. The MBDL receiver is evaluated in terms of uncoded Symbol Error Rate (SER), throughput performance through Link-Level Simulations (LLS) and average training overhead. Results reveal that the MBDL outperforms by a significant margin the SIC receiver with imperfect CSIR.
  arXiv  Detail & Related papers  (2022-05-02T12:23:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.