Information decomposition in complex systems via machine learning

• URL: http://arxiv.org/abs/2307.04755v2
• Date: Tue, 19 Mar 2024 00:36:47 GMT
• Title: Information decomposition in complex systems via machine learning
• Authors: Kieran A. Murphy, Dani S. Bassett,
• Abstract summary: We use machine learning to decompose the information contained in a set of measurements by jointly optimizing a lossy compression of each measurement. We focus our analysis on two paradigmatic complex systems: a circuit and an amorphous material undergoing plastic deformation.
• Score: 4.189643331553922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of linking variation across scales of a system due to its independence of functional relationship between observables. However, characterizing the manner in which information is distributed across a set of observables is computationally challenging and generally infeasible beyond a handful of measurements. Here we propose a practical and general methodology that uses machine learning to decompose the information contained in a set of measurements by jointly optimizing a lossy compression of each measurement. Guided by the distributed information bottleneck as a learning objective, the information decomposition identifies the variation in the measurements of the system state most relevant to specified macroscale behavior. We focus our analysis on two paradigmatic complex systems: a Boolean circuit and an amorphous material undergoing plastic deformation. In both examples, the large amount of entropy of the system state is decomposed, bit by bit, in terms of what is most related to macroscale behavior. The identification of meaningful variation in data, with the full generality brought by information theory, is made practical for studying the connection between micro- and macroscale structure in complex systems.

Related papers

• Targeted Cause Discovery with Data-Driven Learning [66.86881771339145]
  We propose a novel machine learning approach for inferring causal variables of a target variable from observations. We employ a neural network trained to identify causality through supervised learning on simulated data. Empirical results demonstrate the effectiveness of our method in identifying causal relationships within large-scale gene regulatory networks.
  arXiv  Detail & Related papers  (2024-08-29T02:21:11Z)
• On the Role of Information Structure in Reinforcement Learning for Partially-Observable Sequential Teams and Games [55.2480439325792]
  In a sequential decision-making problem, the information structure is the description of how events in the system occurring at different points in time affect each other. By contrast, real-world sequential decision-making problems typically involve a complex and time-varying interdependence of system variables. We formalize a novel reinforcement learning model which explicitly represents the information structure.
  arXiv  Detail & Related papers  (2024-03-01T21:28:19Z)
• Towards Information Theory-Based Discovery of Equivariances [1.2289361708127875]
  The presence of symmetries imposes a stringent set of constraints on a system. In parallel, principled models of complexity-constrained learning and behaviour make increasing use of information-theoretic methods. We propose a novel variant of the Information Bottleneck principle, which has served as a productive basis for many principled studies of learning and information-constrained adaptive behaviour.
  arXiv  Detail & Related papers  (2023-10-25T11:19:40Z)
• Interactive System-wise Anomaly Detection [66.3766756452743]
  Anomaly detection plays a fundamental role in various applications. It is challenging for existing methods to handle the scenarios where the instances are systems whose characteristics are not readily observed as data. We develop an end-to-end approach which includes an encoder-decoder module that learns system embeddings.
  arXiv  Detail & Related papers  (2023-04-21T02:20:24Z)
• Graph-informed simulation-based inference for models of active matter [5.533353383316288]
  We show that simulation-based inference can be used to robustly infer active matter parameters from system observations. Our work highlights that high-level system information is contained within the relational structure of a collective system.
  arXiv  Detail & Related papers  (2023-04-05T09:39:17Z)
• Causality-Based Multivariate Time Series Anomaly Detection [63.799474860969156]
  We formulate the anomaly detection problem from a causal perspective and view anomalies as instances that do not follow the regular causal mechanism to generate the multivariate data. We then propose a causality-based anomaly detection approach, which first learns the causal structure from data and then infers whether an instance is an anomaly relative to the local causal mechanism. We evaluate our approach with both simulated and public datasets as well as a case study on real-world AIOps applications.
  arXiv  Detail & Related papers  (2022-06-30T06:00:13Z)
• The Distributed Information Bottleneck reveals the explanatory structure of complex systems [1.52292571922932]
  The Information Bottleneck (IB) is an information theoretic framework for understanding a relationship between an input and an output. We show that a crucial modification -- distributing bottlenecks across multiple components of the input -- opens fundamentally new avenues for interpretable deep learning in science. We demonstrate the Distributed IB's explanatory utility in systems drawn from applied mathematics and condensed matter physics.
  arXiv  Detail & Related papers  (2022-04-15T17:59:35Z)
• Tracing Information Flow from Open Quantum Systems [52.77024349608834]
  We use photons in a waveguide array to implement a quantum simulation of the coupling of a qubit with a low-dimensional discrete environment. Using the trace distance between quantum states as a measure of information, we analyze different types of information transfer.
  arXiv  Detail & Related papers  (2021-03-22T16:38:31Z)
• On the connection between microscopic description and memory effects in open quantum system dynamics [0.0]
  We investigate the role played by the system-environment correlations and the environmental evolution in the flow of information. Our analysis clarifies how the interplay between system-environment correlations and environmental-state distinguishability can lead to the same information flow from and toward the open system.
  arXiv  Detail & Related papers  (2021-01-18T19:01:19Z)
• Information Theory Measures via Multidimensional Gaussianization [7.788961560607993]
  Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications. However, obtaining information from multidimensional data is a challenging problem due to the curse of dimensionality.
  arXiv  Detail & Related papers  (2020-10-08T07:22:16Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.