Neural Communication Systems with Bandwidth-limited Channel

• URL: http://arxiv.org/abs/2003.13367v2
• Date: Wed, 1 Apr 2020 09:56:13 GMT
• Title: Neural Communication Systems with Bandwidth-limited Channel
• Authors: Karen Ullrich, Fabio Viola, Danilo Jimenez Rezende
• Abstract summary: Reliably transmitting messages despite information loss is a core problem of information theory. In this study we consider learning coding with the bandwidth-limited channel (BWLC)
• Score: 9.332315420944836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reliably transmitting messages despite information loss due to a noisy channel is a core problem of information theory. One of the most important aspects of real world communication, e.g. via wifi, is that it may happen at varying levels of information transfer. The bandwidth-limited channel models this phenomenon. In this study we consider learning coding with the bandwidth-limited channel (BWLC). Recently, neural communication models such as variational autoencoders have been studied for the task of source compression. We build upon this work by studying neural communication systems with the BWLC. Specifically,we find three modelling choices that are relevant under expected information loss. First, instead of separating the sub-tasks of compression (source coding) and error correction (channel coding), we propose to model both jointly. Framing the problem as a variational learning problem, we conclude that joint systems outperform their separate counterparts when coding is performed by flexible learnable function approximators such as neural networks. To facilitate learning, we introduce a differentiable and computationally efficient version of the bandwidth-limited channel. Second, we propose a design to model missing information with a prior, and incorporate this into the channel model. Finally, sampling from the joint model is improved by introducing auxiliary latent variables in the decoder. Experimental results justify the validity of our design decisions through improved distortion and FID scores.

Related papers

• Learning Robust Representations for Communications over Noisy Channels [0.44998333629984877]
  This work relies solely on the tools of information theory and machine learning. We investigate the impact of using various cost functions based on mutual information to generate robust representations for transmission under strict power constraints.
  arXiv  Detail & Related papers  (2024-09-02T10:07:32Z)
• Modeling of Time-varying Wireless Communication Channel with Fading and Shadowing [0.0]
  We propose a new approach that combines a deep learning neural network with a mixture density network model to derive the conditional probability density function of receiving power. Experiments on Nakagami fading channel model and Log-normal shadowing channel model with path loss and noise show that the new approach is more statistically accurate, faster, and more robust than the previous deep learning-based channel models.
  arXiv  Detail & Related papers  (2024-05-13T21:30:50Z)
• Knowledge Distillation Based Semantic Communications For Multiple Users [10.770552656390038]
  We consider the semantic communication (SemCom) system with multiple users, where there is a limited number of training samples and unexpected interference. We propose a knowledge distillation (KD) based system where Transformer based encoder-decoder is implemented as the semantic encoder-decoder and fully connected neural networks are implemented as the channel encoder-decoder. Numerical results demonstrate that KD significantly improves the robustness and the generalization ability when applied to unexpected interference, and it reduces the performance loss when compressing the model size.
  arXiv  Detail & Related papers  (2023-11-23T03:28:14Z)
• Semantics Alignment via Split Learning for Resilient Multi-User Semantic Communication [56.54422521327698]
  Recent studies on semantic communication rely on neural network (NN) based transceivers such as deep joint source and channel coding (DeepJSCC) Unlike traditional transceivers, these neural transceivers are trainable using actual source data and channels, enabling them to extract and communicate semantics. We propose a distributed learning based solution, which leverages split learning (SL) and partial NN fine-tuning techniques.
  arXiv  Detail & Related papers  (2023-10-13T20:29:55Z)
• Dynamic Encoding and Decoding of Information for Split Learning in Mobile-Edge Computing: Leveraging Information Bottleneck Theory [1.1151919978983582]
  Split learning is a privacy-preserving distributed learning paradigm in which an ML model is split into two parts (i.e., an encoder and a decoder) In mobile-edge computing, network functions can be trained via split learning where an encoder resides in a user equipment (UE) and a decoder resides in the edge network. We present a new framework and training mechanism to enable a dynamic balancing of the transmission resource consumption with the informativeness of the shared latent representations.
  arXiv  Detail & Related papers  (2023-09-06T07:04:37Z)
• High Fidelity Neural Audio Compression [92.4812002532009]
  We introduce a state-of-the-art real-time, high-fidelity, audio leveraging neural networks. It consists in a streaming encoder-decoder architecture with quantized latent space trained in an end-to-end fashion. We simplify and speed-up the training by using a single multiscale spectrogram adversary.
  arXiv  Detail & Related papers  (2022-10-24T17:52:02Z)
• Discrete Key-Value Bottleneck [95.61236311369821]
  Deep neural networks perform well on classification tasks where data streams are i.i.d. and labeled data is abundant. One powerful approach that has addressed this challenge involves pre-training of large encoders on volumes of readily available data, followed by task-specific tuning. Given a new task, however, updating the weights of these encoders is challenging as a large number of weights needs to be fine-tuned, and as a result, they forget information about the previous tasks. We propose a model architecture to address this issue, building upon a discrete bottleneck containing pairs of separate and learnable key-value codes.
  arXiv  Detail & Related papers  (2022-07-22T17:52:30Z)
• Error Correction Code Transformer [92.10654749898927]
  We propose to extend for the first time the Transformer architecture to the soft decoding of linear codes at arbitrary block lengths. We encode each channel's output dimension to high dimension for better representation of the bits information to be processed separately. The proposed approach demonstrates the extreme power and flexibility of Transformers and outperforms existing state-of-the-art neural decoders by large margins at a fraction of their time complexity.
  arXiv  Detail & Related papers  (2022-03-27T15:25:58Z)
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network [79.04274563889548]
  We propose SignalNet, a neural network architecture that detects the number of sinusoids and estimates their parameters from quantized in-phase and quadrature samples. We introduce a worst-case learning threshold for comparing the results of our network relative to the underlying data distributions. In simulation, we find that our algorithm is always able to surpass the threshold for three-bit data but often cannot exceed the threshold for one-bit data.
  arXiv  Detail & Related papers  (2021-06-10T04:21:20Z)
• A Variational Auto-Encoder Approach for Image Transmission in Wireless Channel [4.82810058837951]
  We investigate the performance of variational auto-encoders and compare the results with standard auto-encoders. Our experiments demonstrate that the SSIM metric visually improves the quality of the reconstructed images at the receiver.
  arXiv  Detail & Related papers  (2020-10-08T13:35:38Z)
• Data-Driven Symbol Detection via Model-Based Machine Learning [117.58188185409904]
  We review a data-driven framework to symbol detection design which combines machine learning (ML) and model-based algorithms. In this hybrid approach, well-known channel-model-based algorithms are augmented with ML-based algorithms to remove their channel-model-dependence. Our results demonstrate that these techniques can yield near-optimal performance of model-based algorithms without knowing the exact channel input-output statistical relationship.
  arXiv  Detail & Related papers  (2020-02-14T06:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.