A Perspective on Neural Capacity Estimation: Viability and Reliability

• URL: http://arxiv.org/abs/2203.11793v1
• Date: Tue, 22 Mar 2022 14:55:31 GMT
• Title: A Perspective on Neural Capacity Estimation: Viability and Reliability
• Authors: Farhad Mirkarimi, Stefano Rini
• Abstract summary: We study the performance of neural mutual information estimators (NMIE) proposed in the literature when applied to the capacity estimation problem. For the NMIE above, capacity estimation relies on two deep neural networks (DNN) We benchmark these NMIE in three scenarios: (i) AWGN channel capacity estimation and (ii) channels with unknown capacity and continuous inputs.
• Score: 9.251773744318118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowledge of closed-form distribution of the data. This class of estimators is referred to as neural mutual information estimators (NMIE). In this paper, we investigate the performance of different NMIE proposed in the literature when applied to the capacity estimation problem. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE). For the NMIE above, capacity estimation relies on two deep neural networks (DNN): (i) one DNN generates samples from a distribution that is learned, and (ii) a DNN to estimate the MI between the channel input and the channel output. We benchmark these NMIE in three scenarios: (i) AWGN channel capacity estimation and (ii) channels with unknown capacity and continuous inputs i.e., optical intensity and peak-power constrained AWGN channel (iii) channels with unknown capacity and a discrete number of mass points i.e., Poisson channel. Additionally, we also (iv) consider the extension to the MAC capacity problem by considering the AWGN and optical MAC models.

Related papers

• Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy [55.014926694758195]
  Entropy and mutual information in neural networks provide rich information on the learning process. We leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. We show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data.
  arXiv  Detail & Related papers  (2023-12-04T01:32:42Z)
• Patch-CNN: Training data-efficient deep learning for high-fidelity diffusion tensor estimation from minimal diffusion protocols [3.0416974614291226]
  We propose a new method, Patch-CNN, for diffusion tensor (DT) estimation from only six-direction diffusion weighted images (DWI) Compared with image-wise FCNs, the minimal kernel vastly reduces training data demand. The improved fibre orientation estimation is shown to produce improved tractogram.
  arXiv  Detail & Related papers  (2023-07-03T20:39:48Z)
• Efficient Uncertainty Estimation in Spiking Neural Networks via MC-dropout [3.692069129522824]
  Spiking neural networks (SNNs) have gained attention as models of sparse and event-driven communication of biological neurons. We propose an efficient Monte Carlo(MC)-dropout based approach for uncertainty estimation in SNNs.
  arXiv  Detail & Related papers  (2023-04-20T10:05:57Z)
• Variational Neural Networks [88.24021148516319]
  We propose a method for uncertainty estimation in neural networks called Variational Neural Network (VNN) VNN generates parameters for the output distribution of a layer by transforming its inputs with learnable sub-layers. In uncertainty quality estimation experiments, we show that VNNs achieve better uncertainty quality than Monte Carlo Dropout or Bayes By Backpropagation methods.
  arXiv  Detail & Related papers  (2022-07-04T15:41:02Z)
• Deep Learning-Based Synchronization for Uplink NB-IoT [72.86843435313048]
  We propose a neural network (NN)-based algorithm for device detection and time of arrival (ToA) estimation for the narrowband physical random-access channel (NPRACH) of narrowband internet of things (NB-IoT) The introduced NN architecture leverages residual convolutional networks as well as knowledge of the preamble structure of the 5G New Radio (5G NR) specifications.
  arXiv  Detail & Related papers  (2022-05-22T12:16:43Z)
• Neural Capacity Estimators: How Reliable Are They? [14.904387585122851]
  We study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and information directed neural estimator (DINE) We evaluate these algorithms in terms of their ability to learn the input distributions that are capacity approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel.
  arXiv  Detail & Related papers  (2021-11-14T18:14:53Z)
• Learning to Perform Downlink Channel Estimation in Massive MIMO Systems [72.76968022465469]
  We study downlink (DL) channel estimation in a Massive multiple-input multiple-output (MIMO) system. A common approach is to use the mean value as the estimate, motivated by channel hardening. We propose two novel estimation methods.
  arXiv  Detail & Related papers  (2021-09-06T13:42:32Z)
• Learning to Estimate RIS-Aided mmWave Channels [50.15279409856091]
  We focus on uplink cascaded channel estimation, where known and fixed base station combining and RIS phase control matrices are considered for collecting observations. To boost the estimation performance and reduce the training overhead, the inherent channel sparsity of mmWave channels is leveraged in the deep unfolding method. It is verified that the proposed deep unfolding network architecture can outperform the least squares (LS) method with a relatively smaller training overhead and online computational complexity.
  arXiv  Detail & Related papers  (2021-07-27T06:57:56Z)
• State and Topology Estimation for Unobservable Distribution Systems using Deep Neural Networks [8.673621107750652]
  Time-synchronized state estimation for reconfigurable distribution networks is challenging because of limited real-time observability. This paper formulates a deep learning (DL)-based approach for topology identification (TI) and unbalanced three-phase distribution system state estimation (DSSE) Two deep neural networks (DNNs) are trained to operate in a sequential manner for implementing TI and DSSE for systems that are incompletely observed by synchrophasor measurement devices (SMDs)
  arXiv  Detail & Related papers  (2021-04-15T02:46:50Z)
• Deep Networks for Direction-of-Arrival Estimation in Low SNR [89.45026632977456]
  We introduce a Convolutional Neural Network (CNN) that is trained from mutli-channel data of the true array manifold matrix. We train a CNN in the low-SNR regime to predict DoAs across all SNRs. Our robust solution can be applied in several fields, ranging from wireless array sensors to acoustic microphones or sonars.
  arXiv  Detail & Related papers  (2020-11-17T12:52:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.