High-dimensional inference: a statistical mechanics perspective

• URL: http://arxiv.org/abs/2010.14863v1
• Date: Wed, 28 Oct 2020 10:17:21 GMT
• Title: High-dimensional inference: a statistical mechanics perspective
• Authors: Jean Barbier
• Abstract summary: Statistical inference is the science of drawing conclusions about some system from data. It is by now clear that there are many connections between inference and statistical physics. This article has been published in the issue on artificial intelligence of Ithaca, an Italian popularization-of-science journal.
• Score: 11.532173708183166
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Statistical inference is the science of drawing conclusions about some system from data. In modern signal processing and machine learning, inference is done in very high dimension: very many unknown characteristics about the system have to be deduced from a lot of high-dimensional noisy data. This "high-dimensional regime" is reminiscent of statistical mechanics, which aims at describing the macroscopic behavior of a complex system based on the knowledge of its microscopic interactions. It is by now clear that there are many connections between inference and statistical physics. This article aims at emphasizing some of the deep links connecting these apparently separated disciplines through the description of paradigmatic models of high-dimensional inference in the language of statistical mechanics. This article has been published in the issue on artificial intelligence of Ithaca, an Italian popularization-of-science journal. The selected topics and references are highly biased and not intended to be exhaustive in any ways. Its purpose is to serve as introduction to statistical mechanics of inference through a very specific angle that corresponds to my own tastes and limited knowledge.

Related papers

• Unified Explanations in Machine Learning Models: A Perturbation Approach [0.0]
  Inconsistencies between XAI and modeling techniques can have the undesirable effect of casting doubt upon the efficacy of these explainability approaches. We propose a systematic, perturbation-based analysis against a popular, model-agnostic method in XAI, SHapley Additive exPlanations (Shap) We devise algorithms to generate relative feature importance in settings of dynamic inference amongst a suite of popular machine learning and deep learning methods, and metrics that allow us to quantify how well explanations generated under the static case hold.
  arXiv  Detail & Related papers  (2024-05-30T16:04:35Z)
• Rethinking Explainable Machine Learning as Applied Statistics [9.03268085547399]
  We argue that explainable machine learning needs to recognize its parallels with applied statistics. The fact that this is scarcely being discussed in research papers is one of the main drawbacks of the current literature.
  arXiv  Detail & Related papers  (2024-02-05T10:36:48Z)
• Large Language Models for Scientific Synthesis, Inference and Explanation [56.41963802804953]
  We show how large language models can perform scientific synthesis, inference, and explanation. We show that the large language model can augment this "knowledge" by synthesizing from the scientific literature. This approach has the further advantage that the large language model can explain the machine learning system's predictions.
  arXiv  Detail & Related papers  (2023-10-12T02:17:59Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Statistical inference as Green's functions [0.0]
  We show that statistical inference has rigorous scientific description for long sequence of exchangeable binary random variables. Our finding is the answer to the normative and foundational issue in science, and its significance will be far-reaching in all pure and applied fields.
  arXiv  Detail & Related papers  (2022-05-23T14:51:32Z)
• Quantifying Relevance in Learning and Inference [0.0]
  We review recent progress on understanding learning, based on the notion of "relevance" These are ideal limits of samples and of machines, that contain the maximal amount of information about the unknown generative process. Maximally informative samples are characterised by a power-law frequency distribution (statistical criticality) and optimal learning machines by an anomalously large susceptibility.
  arXiv  Detail & Related papers  (2022-02-01T11:16:04Z)
• Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks [73.94290462239061]
  We propose to combine symbolism and connectionism principles by using neural networks to derive a discrete representation. By designing an interactive environment and task, we demonstrated that machines could generate a spontaneous, flexible, and semantic language.
  arXiv  Detail & Related papers  (2022-01-14T14:54:58Z)
• Ranking the information content of distance measures [61.754016309475745]
  We introduce a statistical test that can assess the relative information retained when using two different distance measures. This in turn allows finding the most informative distance measure out of a pool of candidates.
  arXiv  Detail & Related papers  (2021-04-30T15:57:57Z)
• Dominant motion identification of multi-particle system using deep learning from video [0.0]
  In this work, we provide a deep-learning framework that extracts relevant information from real-world videos of highly systems. We demonstrate this approach on videos of confined multi-agent/particle systems of ants, termites, fishes. Furthermore, we explore how these seemingly diverse systems have predictable underlying behavior.
  arXiv  Detail & Related papers  (2021-04-26T17:10:56Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Gaussian Process States: A data-driven representation of quantum many-body physics [59.7232780552418]
  We present a novel, non-parametric form for compactly representing entangled many-body quantum states. The state is found to be highly compact, systematically improvable and efficient to sample. It is also proven to be a universal approximator' for quantum states, able to capture any entangled many-body state with increasing data set size.
  arXiv  Detail & Related papers  (2020-02-27T15:54:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.