Semantic Multi-Resolution Communications

• URL: http://arxiv.org/abs/2308.11604v1
• Date: Tue, 22 Aug 2023 17:52:44 GMT
• Title: Semantic Multi-Resolution Communications
• Authors: Matin Mortaheb, Mohammad A. Amir Khojastepour, Srimat T. Chakradhar, Sennur Ulukus
• Abstract summary: We propose a novel deep learning multi-resolution JSCC framework inspired by the concept of multi-task learning (MTL) This framework excels at encoding data for different resolutions through hierarchical layers and effectively decodes it by leveraging both current and past layers of encoded data. This framework holds great potential for semantic communication, where the objective extends beyond data reconstruction to preserving specific semantic attributes.
• Score: 31.285983939625098
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning based joint source-channel coding (JSCC) has demonstrated significant advancements in data reconstruction compared to separate source-channel coding (SSCC). This superiority arises from the suboptimality of SSCC when dealing with finite block-length data. Moreover, SSCC falls short in reconstructing data in a multi-user and/or multi-resolution fashion, as it only tries to satisfy the worst channel and/or the highest quality data. To overcome these limitations, we propose a novel deep learning multi-resolution JSCC framework inspired by the concept of multi-task learning (MTL). This proposed framework excels at encoding data for different resolutions through hierarchical layers and effectively decodes it by leveraging both current and past layers of encoded data. Moreover, this framework holds great potential for semantic communication, where the objective extends beyond data reconstruction to preserving specific semantic attributes throughout the communication process. These semantic features could be crucial elements such as class labels, essential for classification tasks, or other key attributes that require preservation. Within this framework, each level of encoded data can be carefully designed to retain specific data semantics. As a result, the precision of a semantic classifier can be progressively enhanced across successive layers, emphasizing the preservation of targeted semantics throughout the encoding and decoding stages. We conduct experiments on MNIST and CIFAR10 dataset. The experiment with both datasets illustrates that our proposed method is capable of surpassing the SSCC method in reconstructing data with different resolutions, enabling the extraction of semantic features with heightened confidence in successive layers. This capability is particularly advantageous for prioritizing and preserving more crucial semantic features within the datasets.

Related papers

• Multimodal generative semantic communication based on latent diffusion model [13.035207938169844]
  This paper introduces a multimodal generative semantic communication framework named mm-GESCO. The framework ingests streams of visible and infrared modal image data, generates fused semantic segmentation maps, and transmits them. At the receiving end, the framework can reconstruct the original multimodal images based on the semantic maps.
  arXiv  Detail & Related papers  (2024-08-10T06:23:41Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• Joint Task and Data Oriented Semantic Communications: A Deep Separate Source-channel Coding Scheme [17.4244108919728]
  To serve both the data transmission and semantic tasks, joint data compression and semantic analysis has become pivotal issue in semantic communications. This paper proposes a deep separate source-channel coding framework for the joint task and data oriented semantic communications. An iterative training algorithm is proposed to tackle the overfitting issue of deep learning models.
  arXiv  Detail & Related papers  (2023-02-27T08:34:37Z)
• Dataset Condensation with Latent Space Knowledge Factorization and Sharing [73.31614936678571]
  We introduce a novel approach for solving dataset condensation problem by exploiting the regularity in a given dataset. Instead of condensing the dataset directly in the original input space, we assume a generative process of the dataset with a set of learnable codes. We experimentally show that our method achieves new state-of-the-art records by significant margins on various benchmark datasets.
  arXiv  Detail & Related papers  (2022-08-21T18:14:08Z)
• Continual Variational Autoencoder Learning via Online Cooperative Memorization [11.540150938141034]
  Variational Autoencoders (VAE) have been successfully used in continual learning classification tasks. However, their ability to generate images with specifications corresponding to the classes and databases learned during Continual Learning is not well understood. We develop a new theoretical framework that formulates CL as a dynamic optimal transport problem. We then propose a novel memory buffering approach, namely the Online Cooperative Memorization (OCM) framework.
  arXiv  Detail & Related papers  (2022-07-20T18:19:27Z)
• Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation [5.281694565226513]
  We apply contrastive learning to enhance the discriminative power of the multi-scale features extracted by semantic segmentation networks. By first mapping the encoder's multi-scale representations to a common feature space, we instantiate a novel form of supervised local-global constraint.
  arXiv  Detail & Related papers  (2022-03-25T01:24:24Z)
• Specificity-preserving RGB-D Saliency Detection [103.3722116992476]
  We propose a specificity-preserving network (SP-Net) for RGB-D saliency detection. Two modality-specific networks and a shared learning network are adopted to generate individual and shared saliency maps. Experiments on six benchmark datasets demonstrate that our SP-Net outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-08-18T14:14:22Z)
• A Holistically-Guided Decoder for Deep Representation Learning with Applications to Semantic Segmentation and Object Detection [74.88284082187462]
  One common strategy is to adopt dilated convolutions in the backbone networks to extract high-resolution feature maps. We propose one novel holistically-guided decoder which is introduced to obtain the high-resolution semantic-rich feature maps.
  arXiv  Detail & Related papers  (2020-12-18T10:51:49Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• Sequential Hierarchical Learning with Distribution Transformation for Image Super-Resolution [83.70890515772456]
  We build a sequential hierarchical learning super-resolution network (SHSR) for effective image SR. We consider the inter-scale correlations of features, and devise a sequential multi-scale block (SMB) to progressively explore the hierarchical information. Experiment results show SHSR achieves superior quantitative performance and visual quality to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-07-19T01:35:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.