Local Approximation of Secrecy Capacity

• URL: http://arxiv.org/abs/2403.13345v1
• Date: Wed, 20 Mar 2024 07:07:13 GMT
• Title: Local Approximation of Secrecy Capacity
• Authors: Emmanouil M. Athanasakos, Nicholas Kalouptsidis, Hariprasad Manjunath,
• Abstract summary: We investigate a scenario of efficiently transmitting a small amount of information subject to compression rate and secrecy constraints. We transform the information-theoretic problem into a linear algebra problem and obtain the perturbed probability distributions such that secrecy is achievable.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper uses Euclidean Information Theory (EIT) to analyze the wiretap channel. We investigate a scenario of efficiently transmitting a small amount of information subject to compression rate and secrecy constraints. We transform the information-theoretic problem into a linear algebra problem and obtain the perturbed probability distributions such that secrecy is achievable. Local approximations are being used in order to obtain an estimate of the secrecy capacity by solving a generalized eigenvalue problem.

Related papers

• Local Information-Theoretic Security via Euclidean Geometry [0.0]
  This paper introduces a methodology based on Euclidean information theory to investigate local properties of secure communication over memoryless wiretap channels.<n>We analyze both the information leakage to an eavesdropper informational efficiency and the largest generalized cost of a secret message.
  arXiv  Detail & Related papers  (2025-10-15T15:19:59Z)
• The Condition Number as a Scale-Invariant Proxy for Information Encoding in Neural Units [0.0]
  We argue that a high condition number, though not sufficient for effective knowledge encoding, may indicate that the unit has learned to selectively amplify and compress information.<n>We present a practical case study where these principles are applied to guide selective fine-tuning of a multimodal Large Language Model.
  arXiv  Detail & Related papers  (2025-06-19T13:06:16Z)
• Reverse em-problem based on Bregman divergence and its application to classical and quantum information theory [53.64687146666141]
  Recent paper introduced an analytical method for calculating the channel capacity without the need for iteration. We turn our attention to the reverse em-problem, proposed by Toyota. We derive a non-iterative formula for the reverse em-problem.
  arXiv  Detail & Related papers  (2024-03-14T10:20:28Z)
• Information Theoretical Importance Sampling Clustering [18.248246885248733]
  A current assumption of most clustering methods is that the training data and future data are taken from the same distribution. We propose an information theoretical importance sampling based approach for clustering problems (ITISC) Experiment results on synthetic datasets and a real-world load forecasting problem validate the effectiveness of the proposed model.
  arXiv  Detail & Related papers  (2023-02-09T03:18:53Z)
• STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning [111.75423966239092]
  We propose an exploration incentive in terms of the integral probability metric (IPM) between a current estimate of the transition model and the unknown optimal. Based on KSD, we develop a novel algorithm algo: textbfSTEin information dirtextbfEcted exploration for model-based textbfReinforcement LearntextbfING.
  arXiv  Detail & Related papers  (2023-01-28T00:49:28Z)
• On the Generalization for Transfer Learning: An Information-Theoretic Analysis [8.102199960821165]
  We give an information-theoretic analysis of the generalization error and excess risk of transfer learning algorithms. Our results suggest, perhaps as expected, that the Kullback-Leibler divergenceD(mu|mu')$ plays an important role in the characterizations. We then generalize the mutual information bound with other divergences such as $phi$-divergence and Wasserstein distance.
  arXiv  Detail & Related papers  (2022-07-12T08:20:41Z)
• On Leave-One-Out Conditional Mutual Information For Generalization [122.2734338600665]
  We derive information theoretic generalization bounds for supervised learning algorithms based on a new measure of leave-one-out conditional mutual information (loo-CMI) Contrary to other CMI bounds, our loo-CMI bounds can be computed easily and can be interpreted in connection to other notions such as classical leave-one-out cross-validation. We empirically validate the quality of the bound by evaluating its predicted generalization gap in scenarios for deep learning.
  arXiv  Detail & Related papers  (2022-07-01T17:58:29Z)
• A Bayesian Framework for Information-Theoretic Probing [51.98576673620385]
  We argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. This paper proposes a new framework to measure what we term Bayesian mutual information.
  arXiv  Detail & Related papers  (2021-09-08T18:08:36Z)
• Featurized Density Ratio Estimation [82.40706152910292]
  In our work, we propose to leverage an invertible generative model to map the two distributions into a common feature space prior to estimation. This featurization brings the densities closer together in latent space, sidestepping pathological scenarios where the learned density ratios in input space can be arbitrarily inaccurate. At the same time, the invertibility of our feature map guarantees that the ratios computed in feature space are equivalent to those in input space.
  arXiv  Detail & Related papers  (2021-07-05T18:30:26Z)
• On the Relevance-Complexity Region of Scalable Information Bottleneck [15.314757778110955]
  We study a variation of the problem, called scalable information bottleneck, where the encoder outputs multiple descriptions of the observation. The problem at hand is motivated by some application scenarios that require varying levels of accuracy depending on the allowed level of generalization.
  arXiv  Detail & Related papers  (2020-11-02T22:25:28Z)
• Focus of Attention Improves Information Transfer in Visual Features [80.22965663534556]
  This paper focuses on unsupervised learning for transferring visual information in a truly online setting. The computation of the entropy terms is carried out by a temporal process which yields online estimation of the entropy terms. In order to better structure the input probability distribution, we use a human-like focus of attention model.
  arXiv  Detail & Related papers  (2020-06-16T15:07:25Z)
• Generalization Bounds via Information Density and Conditional Information Density [14.147617330278662]
  We present a general approach, based on an exponential inequality, to derive bounds on the generalization error of randomized learning algorithms. We provide bounds on the average generalization error as well as bounds on its tail probability, for both the PAC-Bayesian and single-draw scenarios.
  arXiv  Detail & Related papers  (2020-05-16T17:04:24Z)
• Generalization Error Bounds via $m$th Central Moments of the Information Density [14.147617330278662]
  We present a general approach to deriving bounds on the generalization error of randomized learning algorithms. Our approach can be used to obtain bounds on the average generalization error as well as bounds on its tail probabilities.
  arXiv  Detail & Related papers  (2020-04-20T09:23:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.