Inferring High-Order Couplings with Neural Networks

• URL: http://arxiv.org/abs/2501.06108v2
• Date: Mon, 10 Feb 2025 15:28:53 GMT
• Title: Inferring High-Order Couplings with Neural Networks
• Authors: Aurélien Decelle, Alfonso de Jesús Navas Gómez, Beatriz Seoane,
• Abstract summary: We introduce a new method that maps Restricted Boltzmann Machines to generalized Potts models, allowing for the extraction of interactions up to any specified order. Our validation on synthetic datasets confirms the method's ability to recover two- and three-body interactions accurately. When applied to protein sequence data, the framework competently reconstructs protein contact maps and provides performance comparable to the best inverse Potts models.
• Score: 3.55026004901472
• License:
• Abstract: Maximum entropy methods, based on the inverse Ising/Potts problem from statistical mechanics, are essential for modeling interactions between pairs of variables in data-driven problems across disciplines such as bioinformatics, ecology, and neuroscience. Despite their considerable success, these methods typically fail to capture higher-order interactions that are often essential for understanding complex systems. Conversely, modern machine learning methods capture these complex interactions, but the computational cost of interpretable frameworks makes them impractical for real-world applications. Restricted Boltzmann Machines (RBMs) provide a computationally efficient way to capture statistical correlations using hidden nodes in a bipartite neural network. In this study, we introduce a new method that maps RBMs to generalized Potts models, allowing for the extraction of interactions up to any specified order. This method utilizes large-$N$ approximations, enabled by the RBM's simple structure, to extract effective many-body couplings with minimal computational effort. Furthermore, we propose a robust framework for extracting higher-order interactions in more complex probabilistic models and a simple gauge-fixing method within the effective many-body Potts model. Our validation on synthetic datasets confirms the method's ability to recover two- and three-body interactions accurately. When applied to protein sequence data, the framework competently reconstructs protein contact maps and provides performance comparable to the best inverse Potts models. These findings confirm that RBMs are an effective and streamlined tool for exploring higher-order interactions within complex systems.

Related papers

• MaD-Scientist: AI-based Scientist solving Convection-Diffusion-Reaction Equations Using Massive PINN-Based Prior Data [22.262191225577244]
  We explore whether a similar approach can be applied to scientific foundation models (SFMs) We collect low-cost physics-informed neural network (PINN)-based approximated prior data in the form of solutions to partial differential equations (PDEs) constructed through an arbitrary linear combination of mathematical dictionaries. We provide experimental evidence on the one-dimensional convection-diffusion-reaction equation, which demonstrate that pre-training remains robust even with approximated prior data.
  arXiv  Detail & Related papers  (2024-10-09T00:52:00Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• Inferring effective couplings with Restricted Boltzmann Machines [3.150368120416908]
  Generative models attempt to encode correlations observed in the data at the level of the Boltzmann weight associated with an energy function in the form of a neural network. We propose a solution by implementing a direct mapping between the Restricted Boltzmann Machine and an effective Ising spin Hamiltonian.
  arXiv  Detail & Related papers  (2023-09-05T14:55:09Z)
• Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactions [1.9457612782595313]
  Non-trivial dependencies between groups of more than two variables can play a significant role in the analysis and modelling of such systems. We introduce a hierarchy of $d$-order ($d geq 2$) interaction measures, increasingly inclusive of possible factorisations of the joint probability distribution. We also establish mathematical links with lattice theory, which elucidate the derivation of the interaction measures and their composite permutation tests.
  arXiv  Detail & Related papers  (2023-06-01T16:59:37Z)
• Variational Hierarchical Mixtures for Probabilistic Learning of Inverse Dynamics [20.953728061894044]
  Well-calibrated probabilistic regression models are a crucial learning component in robotics applications as datasets grow rapidly and tasks become more complex. We consider a probabilistic hierarchical modeling paradigm that combines the benefits of both worlds to deliver computationally efficient representations with inherent complexity regularization. We derive two efficient variational inference techniques to learn these representations and highlight the advantages of hierarchical infinite local regression models.
  arXiv  Detail & Related papers  (2022-11-02T13:54:07Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learning [89.31889875864599]
  We propose an efficient model-based reinforcement learning algorithm for learning in multi-agent systems. Our main theoretical contributions are the first general regret bounds for model-based reinforcement learning for MFC. We provide a practical parametrization of the core optimization problem.
  arXiv  Detail & Related papers  (2021-07-08T18:01:02Z)
• Weighted Ensemble-model and Network Analysis: A method to predict fluid intelligence via naturalistic functional connectivity [2.66512000865131]
  We propose a new method namely Weighted Ensemble-model and Network Analysis. It combines the machine learning and graph theory for improved fluid intelligence prediction. Our proposed methods achieved best performance with 3.85 mean absolute deviation, 0.66 correlation coefficient and 0.42 R-squared coefficient.
  arXiv  Detail & Related papers  (2021-01-06T11:17:49Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)
• Exact representations of many body interactions with RBM neural networks [77.34726150561087]
  We exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators. This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as Pionless EFT in nuclear physics.
  arXiv  Detail & Related papers  (2020-05-07T15:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.