Capacity of Continuous Channels with Memory via Directed Information Neural Estimator

• URL: http://arxiv.org/abs/2003.04179v2
• Date: Sat, 16 May 2020 16:51:15 GMT
• Title: Capacity of Continuous Channels with Memory via Directed Information Neural Estimator
• Authors: Ziv Aharoni and Dor Tsur and Ziv Goldfeld and Haim Henry Permuter
• Abstract summary: This work proposes a novel capacity estimation algorithm that treats the channel as a blackbox' The algorithm has two main ingredients: (i) a neural distribution transformer (NDT) model that shapes a noise variable into the channel input distribution, and (ii) the neural DI estimator (DINE) that estimates the communication rate of the current NDT model.
• Score: 15.372626012233736
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Calculating the capacity (with or without feedback) of channels with memory and continuous alphabets is a challenging task. It requires optimizing the directed information (DI) rate over all channel input distributions. The objective is a multi-letter expression, whose analytic solution is only known for a few specific cases. When no analytic solution is present or the channel model is unknown, there is no unified framework for calculating or even approximating capacity. This work proposes a novel capacity estimation algorithm that treats the channel as a `black-box', both when feedback is or is not present. The algorithm has two main ingredients: (i) a neural distribution transformer (NDT) model that shapes a noise variable into the channel input distribution, which we are able to sample, and (ii) the DI neural estimator (DINE) that estimates the communication rate of the current NDT model. These models are trained by an alternating maximization procedure to both estimate the channel capacity and obtain an NDT for the optimal input distribution. The method is demonstrated on the moving average additive Gaussian noise channel, where it is shown that both the capacity and feedback capacity are estimated without knowledge of the channel transition kernel. The proposed estimation framework opens the door to a myriad of capacity approximation results for continuous alphabet channels that were inaccessible until now.

Related papers

• Maximal-Capacity Discrete Memoryless Channel Identification [37.598696937684245]
  The problem of identifying the channel with the highest capacity among several memoryless channels (DMCs) is considered. A capacity estimator is proposed and tight confidence bounds on the estimator error are derived. A gap-elimination algorithm termed BestChanID is proposed, which is oblivious to the capacity-achieving input distribution. Two additional algorithms NaiveChanSel and MedianChanEl, that output with certain confidence a DMC with capacity close to the maximal, are introduced.
  arXiv  Detail & Related papers  (2024-01-18T18:44:10Z)
• Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion [56.38386580040991]
  Consistency Trajectory Model (CTM) is a generalization of Consistency Models (CM) CTM enables the efficient combination of adversarial training and denoising score matching loss to enhance performance. Unlike CM, CTM's access to the score function can streamline the adoption of established controllable/conditional generation methods.
  arXiv  Detail & Related papers  (2023-10-01T05:07:17Z)
• Data-Driven Upper Bounds on Channel Capacity [4.974890682815778]
  We consider the problem of estimating an upper bound on the capacity of a memoryless channel with unknown alphabet output. A novel-driven algorithm is proposed that exploits dual representation where the minimization over the input distribution is replaced with a reference distribution on the channel output.
  arXiv  Detail & Related papers  (2022-05-13T06:59:31Z)
• Deep Diffusion Models for Robust Channel Estimation [1.7259824817932292]
  We introduce a novel approach for multiple-input multiple-output (MIMO) channel estimation using deep diffusion models. Our method uses a deep neural network that is trained to estimate the gradient of the log-likelihood of wireless channels at any point in high-dimensional space.
  arXiv  Detail & Related papers  (2021-11-16T01:32:11Z)
• Learning to Perform Downlink Channel Estimation in Massive MIMO Systems [72.76968022465469]
  We study downlink (DL) channel estimation in a Massive multiple-input multiple-output (MIMO) system. A common approach is to use the mean value as the estimate, motivated by channel hardening. We propose two novel estimation methods.
  arXiv  Detail & Related papers  (2021-09-06T13:42:32Z)
• Discriminative Mutual Information Estimators for Channel Capacity Learning [1.8275108630751837]
  We propose a novel framework to automatically learn the channel capacity, for any type of memory-less channel. We include the discriminator in a cooperative channel capacity learning framework, referred to as CORTICAL. We prove that a particular choice of cooperative value function solves the channel capacity estimation problem.
  arXiv  Detail & Related papers  (2021-07-07T09:03:40Z)
• Learning Frequency Domain Approximation for Binary Neural Networks [68.79904499480025]
  We propose to estimate the gradient of sign function in the Fourier frequency domain using the combination of sine functions for training BNNs. The experiments on several benchmark datasets and neural architectures illustrate that the binary network learned using our method achieves the state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2021-03-01T08:25:26Z)
• Channel Estimation via Successive Denoising in MIMO OFDM Systems: A Reinforcement Learning Approach [23.57305243608369]
  We present a frequency-domain denoising method based on a reinforcement learning framework. Our algorithm provides a significant improvement over the practical least squares (LS) estimation method.
  arXiv  Detail & Related papers  (2021-01-25T18:33:54Z)
• Deep Networks for Direction-of-Arrival Estimation in Low SNR [89.45026632977456]
  We introduce a Convolutional Neural Network (CNN) that is trained from mutli-channel data of the true array manifold matrix. We train a CNN in the low-SNR regime to predict DoAs across all SNRs. Our robust solution can be applied in several fields, ranging from wireless array sensors to acoustic microphones or sonars.
  arXiv  Detail & Related papers  (2020-11-17T12:52:18Z)
• Operation-Aware Soft Channel Pruning using Differentiable Masks [51.04085547997066]
  We propose a data-driven algorithm, which compresses deep neural networks in a differentiable way by exploiting the characteristics of operations. We perform extensive experiments and achieve outstanding performance in terms of the accuracy of output networks.
  arXiv  Detail & Related papers  (2020-07-08T07:44:00Z)
• Data-Driven Symbol Detection via Model-Based Machine Learning [117.58188185409904]
  We review a data-driven framework to symbol detection design which combines machine learning (ML) and model-based algorithms. In this hybrid approach, well-known channel-model-based algorithms are augmented with ML-based algorithms to remove their channel-model-dependence. Our results demonstrate that these techniques can yield near-optimal performance of model-based algorithms without knowing the exact channel input-output statistical relationship.
  arXiv  Detail & Related papers  (2020-02-14T06:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.