The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions

• URL: http://arxiv.org/abs/2304.14219v4
• Date: Fri, 6 Sep 2024 12:29:56 GMT
• Title: The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions
• Authors: Barış Nakiboğlu, Hao-Chung Cheng,
• Abstract summary: The exact characterization of the slowest decrease of the mutual information with the distance to $Pi_mathcalA$ is determined on small neighborhoods of $Pi_mathcalA$. Results for classical-quantum channels are established under separable output Hilbert space assumption for the quadratic bound and under finite-dimensional output Hilbert space assumption for the exact characterization.
• Score: 6.675805308519987
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough neighborhoods of $\Pi_{\mathcal{A}}$, using an identity due to Tops{\o}e and the Pinsker's inequality, assuming that the input set of the channel is finite and the constraint set $\mathcal{A}$ is polyhedral, i.e., can be described by (possibly multiple but) finitely many linear constraints. Counterexamples demonstrating nonexistence of such a quadratic bound are provided for the case of infinitely many linear constraints and the case of infinite input sets. Using Taylor's theorem with the remainder term, rather than the Pinsker's inequality and invoking Moreau's decomposition theorem the exact characterization of the slowest decrease of the mutual information with the distance to $\Pi_{\mathcal{A}}$ is determined on small neighborhoods of $\Pi_{\mathcal{A}}$. Corresponding results for classical-quantum channels are established under separable output Hilbert space assumption for the quadratic bound and under finite-dimensional output Hilbert space assumption for the exact characterization. Implications of these observations for the channel coding problem and applications of the proof techniques to related problems are discussed.

Related papers

• Approximation of optimization problems with constraints through kernel Sum-Of-Squares [77.27820145069515]
  We show that pointwise inequalities are turned into equalities within a class of nonnegative kSoS functions. We also show that focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints.
  arXiv  Detail & Related papers  (2023-01-16T10:30:04Z)
• A Law of Robustness beyond Isoperimetry [84.33752026418045]
  We prove a Lipschitzness lower bound $Omega(sqrtn/p)$ of robustness of interpolating neural network parameters on arbitrary distributions. We then show the potential benefit of overparametrization for smooth data when $n=mathrmpoly(d)$. We disprove the potential existence of an $O(1)$-Lipschitz robust interpolating function when $n=exp(omega(d))$.
  arXiv  Detail & Related papers  (2022-02-23T16:10:23Z)
• A lower bound on the space overhead of fault-tolerant quantum computation [51.723084600243716]
  The threshold theorem is a fundamental result in the theory of fault-tolerant quantum computation. We prove an exponential upper bound on the maximal length of fault-tolerant quantum computation with amplitude noise.
  arXiv  Detail & Related papers  (2022-01-31T22:19:49Z)
• Perturbational Complexity by Distribution Mismatch: A Systematic Analysis of Reinforcement Learning in Reproducing Kernel Hilbert Space [0.76146285961466]
  We analyze reinforcement learning in a general reproducing kernel Hilbert space (RKHS) We consider a family of Markov decision processes $mathcalM$ of which the reward functions lie in the unit ball of an RKHS. We show that when the reward functions lie in a high dimensional RKHS, even if the transition probability is known and the action space is finite, it is still possible for RL problems to suffer from the curse of dimensionality.
  arXiv  Detail & Related papers  (2021-11-05T12:46:04Z)
• Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields [53.31927549039624]
  We show that a piecewise discretization preserves better contrast from existing discretization problems. We apply this theory to the problem of matching two images.
  arXiv  Detail & Related papers  (2021-07-13T12:31:06Z)
• Nonparametric approximation of conditional expectation operators [0.3655021726150368]
  We investigate the approximation of the $L2$-operator defined by $[Pf](x) := mathbbE[ f(Y) mid X = x ]$ under minimal assumptions. We prove that $P$ can be arbitrarily well approximated in operator norm by Hilbert-Schmidt operators acting on a reproducing kernel space.
  arXiv  Detail & Related papers  (2020-12-23T19:06:12Z)
• Approximation of BV functions by neural networks: A regularity theory approach [0.0]
  We are concerned with the approximation of functions by single hidden layer neural networks with ReLU activation functions on the unit circle. We first study the convergence to equilibrium of the gradient flow associated with the cost function with a penalization. As our penalization biases the weights to be bounded, this leads us to study how well a network with bounded weights can approximate a given function of bounded variation.
  arXiv  Detail & Related papers  (2020-12-15T13:58:44Z)
• Scattering data and bound states of a squeezed double-layer structure [77.34726150561087]
  A structure composed of two parallel homogeneous layers is studied in the limit as their widths $l_j$ and $l_j$, and the distance between them $r$ shrinks to zero simultaneously. The existence of non-trivial bound states is proven in the squeezing limit, including the particular example of the squeezed potential in the form of the derivative of Dirac's delta function. The scenario how a single bound state survives in the squeezed system from a finite number of bound states in the finite system is described in detail.
  arXiv  Detail & Related papers  (2020-11-23T14:40:27Z)
• Linear Optimal Transport Embedding: Provable Wasserstein classification for certain rigid transformations and perturbations [79.23797234241471]
  Discriminating between distributions is an important problem in a number of scientific fields. The Linear Optimal Transportation (LOT) embeds the space of distributions into an $L2$-space. We demonstrate the benefits of LOT on a number of distribution classification problems.
  arXiv  Detail & Related papers  (2020-08-20T19:09:33Z)
• Inexact and Stochastic Generalized Conditional Gradient with Augmented Lagrangian and Proximal Step [2.0196229393131726]
  We analyze inexact and versions of the CGALP algorithm developed in the authors' previous paper. This allows one to compute some gradients, terms, and/or linear minimization oracles in an inexact fashion. We show convergence of the Lagrangian to an optimum and feasibility of the affine constraint.
  arXiv  Detail & Related papers  (2020-05-11T14:52:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.