Information Geometry for the Working Information Theorist

• URL: http://arxiv.org/abs/2310.03884v1
• Date: Thu, 5 Oct 2023 20:36:10 GMT
• Title: Information Geometry for the Working Information Theorist
• Authors: Kumar Vijay Mishra, M. Ashok Kumar and Ting-Kam Leonard Wong
• Abstract summary: Information geometry is an interdisciplinary field that finds applications in diverse areas such as radar sensing, array signal processing, quantum physics, deep learning, and optimal transport. This article presents an overview of essential information geometry to initiate an information theorist, who may be unfamiliar with this exciting area of research.
• Score: 18.42675731940218
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information, sufficient statistics, and efficient estimators. Today, information geometry has emerged as an interdisciplinary field that finds applications in diverse areas such as radar sensing, array signal processing, quantum physics, deep learning, and optimal transport. This article presents an overview of essential information geometry to initiate an information theorist, who may be unfamiliar with this exciting area of research. We explain the concepts of divergences on statistical manifolds, generalized notions of distances, orthogonality, and geodesics, thereby paving the way for concrete applications and novel theoretical investigations. We also highlight some recent information-geometric developments, which are of interest to the broader information theory community.

Related papers

• Geometry Distributions [51.4061133324376]
  We propose a novel geometric data representation that models geometry as distributions. Our approach uses diffusion models with a novel network architecture to learn surface point distributions. We evaluate our representation qualitatively and quantitatively across various object types, demonstrating its effectiveness in achieving high geometric fidelity.
  arXiv  Detail & Related papers  (2024-11-25T04:06:48Z)
• Ontology Embedding: A Survey of Methods, Applications and Resources [54.3453925775069]
  Ontologies are widely used for representing domain knowledge and meta data. One straightforward solution is to integrate statistical analysis and machine learning. Numerous papers have been published on embedding, but a lack of systematic reviews hinders researchers from gaining a comprehensive understanding of this field.
  arXiv  Detail & Related papers  (2024-06-16T14:49:19Z)
• A Survey of Geometric Graph Neural Networks: Data Structures, Models and Applications [67.33002207179923]
  This paper presents a survey of data structures, models, and applications related to geometric GNNs. We provide a unified view of existing models from the geometric message passing perspective. We also summarize the applications as well as the related datasets to facilitate later research for methodology development and experimental evaluation.
  arXiv  Detail & Related papers  (2024-03-01T12:13:04Z)
• The Fisher-Rao geometry of CES distributions [50.50897590847961]
  The Fisher-Rao information geometry allows for leveraging tools from differential geometry. We will present some practical uses of these geometric tools in the framework of elliptical distributions.
  arXiv  Detail & Related papers  (2023-10-02T09:23:32Z)
• Symmetry-Informed Geometric Representation for Molecules, Proteins, and Crystalline Materials [66.14337835284628]
  We propose a platform, coined Geom3D, which enables benchmarking the effectiveness of geometric strategies. Geom3D contains 16 advanced symmetry-informed geometric representation models and 14 geometric pretraining methods over 46 diverse datasets.
  arXiv  Detail & Related papers  (2023-06-15T05:37:25Z)
• Recent Advances in Algebraic Geometry and Bayesian Statistics [0.0]
  This article is a review of theoretical advances in the research field of algebraic geometry and Bayesian statistics. Two mathematical solutions and three applications to statistics based on algebraic geometry reported in this article are now being used in many practical fields in data science and artificial intelligence.
  arXiv  Detail & Related papers  (2022-11-18T06:19:05Z)
• Machine Learning Statistical Gravity from Multi-Region Entanglement Entropy [0.0]
  Ryu-Takayanagi formula connects quantum entanglement and geometry. We propose a microscopic model by superimposing entanglement features of an ensemble of random tensor networks of different bond dimensions. We show mutual information can be mediated effectively by geometric fluctuation.
  arXiv  Detail & Related papers  (2021-10-03T22:46:41Z)
• Ranking the information content of distance measures [61.754016309475745]
  We introduce a statistical test that can assess the relative information retained when using two different distance measures. This in turn allows finding the most informative distance measure out of a pool of candidates.
  arXiv  Detail & Related papers  (2021-04-30T15:57:57Z)
• Bayesian Quadrature on Riemannian Data Manifolds [79.71142807798284]
  A principled way to model nonlinear geometric structure inherent in data is provided. However, these operations are typically computationally demanding. In particular, we focus on Bayesian quadrature (BQ) to numerically compute integrals over normal laws. We show that by leveraging both prior knowledge and an active exploration scheme, BQ significantly reduces the number of required evaluations.
  arXiv  Detail & Related papers  (2021-02-12T17:38:04Z)
• Gaussianizing the Earth: Multidimensional Information Measures for Earth Data Analysis [9.464720193746395]
  Information theory is an excellent framework for analyzing Earth system data. It allows us to characterize uncertainty and redundancy, and is universally interpretable. We show how information theory measures can be applied in various Earth system data analysis problems.
  arXiv  Detail & Related papers  (2020-10-13T15:30:34Z)
• Representations, Metrics and Statistics For Shape Analysis of Elastic Graphs [21.597624908203805]
  This paper introduces a far-reaching geometric approach for analyzing shapes of graphical objects, such as road networks, blood vessels, brain fiber tracts, etc. It represents such objects, exhibiting differences in both geometries and topologies, as graphs made of curves with arbitrary shapes (edges) and connected at arbitrary junctions (nodes)
  arXiv  Detail & Related papers  (2020-02-29T16:07:48Z)
• Information geometry in quantum field theory: lessons from simple examples [0.0]
  We show that the symmetries of the physical theory under study play a strong role in the resulting geometry. We discuss the differences that result from placing a metric on the space of theories vs. states. We clarify some misconceptions in the literature pertaining to different notions of flatness associated to metric and non-metric connections.
  arXiv  Detail & Related papers  (2020-01-08T19:00:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.