Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing

• URL: http://arxiv.org/abs/2411.15702v1
• Date: Sun, 24 Nov 2024 04:07:33 GMT
• Title: Editable-DeepSC: Reliable Cross-Modal Semantic Communications for Facial Editing
• Authors: Bin Chen, Wenbo Yu, Qinshan Zhang, Shu-Tao Xia,
• Abstract summary: We propose Editable-DeepSC, a novel cross-modal semantic communication approach for facial editing. Experiments indicate that Editable-DeepSC can achieve superior editings while significantly saving the transmission bandwidth.
• Score: 45.66741599550691
• License:
• Abstract: Real-time computer vision (CV) plays a crucial role in various real-world applications, whose performance is highly dependent on communication networks. Nonetheless, the data-oriented characteristics of conventional communications often do not align with the special needs of real-time CV tasks. To alleviate this issue, the recently emerged semantic communications only transmit task-related semantic information and exhibit a promising landscape to address this problem. However, the communication challenges associated with Semantic Facial Editing, one of the most important real-time CV applications on social media, still remain largely unexplored. In this paper, we fill this gap by proposing Editable-DeepSC, a novel cross-modal semantic communication approach for facial editing. Firstly, we theoretically discuss different transmission schemes that separately handle communications and editings, and emphasize the necessity of Joint Editing-Channel Coding (JECC) via iterative attributes matching, which integrates editings into the communication chain to preserve more semantic mutual information. To compactly represent the high-dimensional data, we leverage inversion methods via pre-trained StyleGAN priors for semantic coding. To tackle the dynamic channel noise conditions, we propose SNR-aware channel coding via model fine-tuning. Extensive experiments indicate that Editable-DeepSC can achieve superior editings while significantly saving the transmission bandwidth, even under high-resolution and out-of-distribution (OOD) settings.

Related papers

• Take What You Need: Flexible Multi-Task Semantic Communications with Channel Adaptation [51.53221300103261]
  This article introduces a novel channel-adaptive and multi-task-aware semantic communication framework based on a masked auto-encoder architecture. A channel-aware extractor is employed to dynamically select relevant information in response to real-time channel conditions. Experimental results demonstrate the superior performance of our framework compared to conventional methods in tasks such as image reconstruction and object detection.
  arXiv  Detail & Related papers  (2025-02-12T09:01:25Z)
• Scene Understanding Enabled Semantic Communication with Open Channel Coding [3.255136948268739]
  Traditional semantic communication faces limitations, including static coding strategies, poor generalization, and reliance on task-specific knowledge bases that hinder adaptability. We propose a novel system combining scene understanding, Large Language Models (LLMs), and open channel coding, named textbfOpenSC. Experimental results show significant improvements in both semantic understanding and efficiency, advancing the potential of adaptive, generalizable semantic communication in 6G networks.
  arXiv  Detail & Related papers  (2025-01-24T14:23:31Z)
• Cross-Layer Encrypted Semantic Communication Framework for Panoramic Video Transmission [11.438045765196332]
  We propose a cross-layer encrypted semantic communication (CLESC) framework for panoramic video transmission. We propose an adaptive cross-layer transmission mechanism that dynamically adjusts CRC, channel coding, and retransmission schemes based on the importance of semantic information. Compared to traditional cross-layer transmission schemes, the CLESC framework can reduce bandwidth consumption by 85%.
  arXiv  Detail & Related papers  (2024-11-19T07:18:38Z)
• Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
  We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework. To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies. We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
  arXiv  Detail & Related papers  (2024-08-29T08:53:26Z)
• Trustworthy Image Semantic Communication with GenAI: Explainablity, Controllability, and Efficiency [59.15544887307901]
  Image semantic communication (ISC) has garnered significant attention for its potential to achieve high efficiency in visual content transmission. Existing ISC systems based on joint source-channel coding face challenges in interpretability, operability, and compatibility. We propose a novel trustworthy ISC framework that employs Generative Artificial Intelligence (GenAI) for multiple downstream inference tasks.
  arXiv  Detail & Related papers  (2024-08-07T14:32:36Z)
• Multi-level Reliability Interface for Semantic Communications over Wireless Networks [5.9056146376982]
  Joint source-channel coding (JSCC) maps source messages directly into channel input symbols. We propose designing the source and channel mappings separately and sequentially via a novel multi-level reliability interface. This work represents an important step towards realizing semantic communications in wireless networks.
  arXiv  Detail & Related papers  (2024-07-07T20:15:10Z)
• Generative AI-aided Joint Training-free Secure Semantic Communications via Multi-modal Prompts [89.04751776308656]
  This paper proposes a GAI-aided SemCom system with multi-model prompts for accurate content decoding. In response to security concerns, we introduce the application of covert communications aided by a friendly jammer.
  arXiv  Detail & Related papers  (2023-09-05T23:24:56Z)
• Semantic-Native Communication: A Simplicial Complex Perspective [50.099494681671224]
  We study semantic communication from a topological space perspective. A transmitter first maps its data into a $k$-order simplicial complex and then learns its high-order correlations. The receiver decodes the structure and infers the missing or distorted data.
  arXiv  Detail & Related papers  (2022-10-30T22:33:44Z)
• Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach [3.983055670167878]
  A low-end edge device transmits the extracted feature vector of a local data sample to a powerful edge server for processing. It is critical to encode the data into an informative and compact representation for low-latency inference given the limited bandwidth. We propose a learning-based communication scheme that jointly optimize feature extraction, source coding, and channel coding.
  arXiv  Detail & Related papers  (2021-02-08T12:53:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.