Inferring Higher-Order Couplings with Neural Networks

• URL: http://arxiv.org/abs/2501.06108v4
• Date: Tue, 08 Jul 2025 16:40:26 GMT
• Title: Inferring Higher-Order Couplings with Neural Networks
• Authors: Aurélien Decelle, Alfonso de Jesús Navas Gómez, Beatriz Seoane,
• Abstract summary: Maximum entropy methods are widely used to model pairwise interactions in complex systems.<n>Modern machine learning methods can model such interactions, but their interpretability often comes at a prohibitive computational cost.<n>In this work, we introduce a method that maps Restricted Boltzmann Machines onto generalized Potts models, enabling the systematic extraction of interactions up to arbitrary order.
• Score: 3.55026004901472
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Maximum entropy methods, rooted in the inverse Ising/Potts problem from statistical physics, are widely used to model pairwise interactions in complex systems across disciplines such as bioinformatics and neuroscience. While successful, these approaches often fail to capture higher-order interactions that are critical for understanding collective behavior. In contrast, modern machine learning methods can model such interactions, but their interpretability often comes at a prohibitive computational cost. Restricted Boltzmann Machines (RBMs) provide a computationally efficient alternative by encoding statistical correlations through hidden units in a bipartite architecture. In this work, we introduce a method that maps RBMs onto generalized Potts models, enabling the systematic extraction of interactions up to arbitrary order. Leveraging large-$N$ approximations, made tractable by the RBM's structure, we extract effective many-body couplings with minimal computational effort. We further propose a robust framework for recovering higher-order interactions in more complex generative models, and introduce a simple gauge-fixing scheme for the effective Potts representation. Validation on synthetic data demonstrates accurate recovery of two- and three-body interactions. Applied to protein sequence data, our method reconstructs contact maps with high fidelity and outperforms state-of-the-art inverse Potts models. These results establish RBMs as a powerful and efficient tool for modeling higher-order structure in high-dimensional categorical data.

Related papers

• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• BrainSymphony: A Transformer-Driven Fusion of fMRI Time Series and Structural Connectivity [2.3486335708866606]
  BrainSymphony is a lightweight, parameter-efficient foundation model for neuroimaging.<n>It achieves state-of-the-art performance while being pre-trained on significantly smaller public datasets.<n>BrainSymphony establishes that architecturally-aware, multimodal models can surpass their larger counterparts.
  arXiv  Detail & Related papers  (2025-06-23T06:00:21Z)
• MaD-Scientist: AI-based Scientist solving Convection-Diffusion-Reaction Equations Using Massive PINN-Based Prior Data [22.262191225577244]
  We explore whether a similar approach can be applied to scientific foundation models (SFMs) We collect low-cost physics-informed neural network (PINN)-based approximated prior data in the form of solutions to partial differential equations (PDEs) constructed through an arbitrary linear combination of mathematical dictionaries. We provide experimental evidence on the one-dimensional convection-diffusion-reaction equation, which demonstrate that pre-training remains robust even with approximated prior data.
  arXiv  Detail & Related papers  (2024-10-09T00:52:00Z)
• Discovering physical laws with parallel combinatorial tree search [57.05912962368898]
  Symbolic regression plays a crucial role in scientific research thanks to its capability of discovering concise and interpretable mathematical expressions from data. Existing algorithms have faced a critical bottleneck of accuracy and efficiency over a decade. We introduce a parallel tree search (PCTS) model to efficiently distill generic mathematical expressions from limited data.
  arXiv  Detail & Related papers  (2024-07-05T10:41:15Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• CORE: Common Random Reconstruction for Distributed Optimization with Provable Low Communication Complexity [110.50364486645852]
  Communication complexity has become a major bottleneck for speeding up training and scaling up machine numbers. We propose Common Om REOm, which can be used to compress information transmitted between machines.
  arXiv  Detail & Related papers  (2023-09-23T08:45:27Z)
• Inferring effective couplings with Restricted Boltzmann Machines [3.150368120416908]
  Generative models attempt to encode correlations observed in the data at the level of the Boltzmann weight associated with an energy function in the form of a neural network. We propose a solution by implementing a direct mapping between the Restricted Boltzmann Machine and an effective Ising spin Hamiltonian.
  arXiv  Detail & Related papers  (2023-09-05T14:55:09Z)
• Interaction Measures, Partition Lattices and Kernel Tests for High-Order Interactions [1.9457612782595313]
  Non-trivial dependencies between groups of more than two variables can play a significant role in the analysis and modelling of such systems. We introduce a hierarchy of $d$-order ($d geq 2$) interaction measures, increasingly inclusive of possible factorisations of the joint probability distribution. We also establish mathematical links with lattice theory, which elucidate the derivation of the interaction measures and their composite permutation tests.
  arXiv  Detail & Related papers  (2023-06-01T16:59:37Z)
• Linear Self-Attention Approximation via Trainable Feedforward Kernel [77.34726150561087]
  In pursuit of faster computation, Efficient Transformers demonstrate an impressive variety of approaches. We aim to expand the idea of trainable kernel methods to approximate the self-attention mechanism of the Transformer architecture.
  arXiv  Detail & Related papers  (2022-11-08T08:14:11Z)
• Variational Hierarchical Mixtures for Probabilistic Learning of Inverse Dynamics [20.953728061894044]
  Well-calibrated probabilistic regression models are a crucial learning component in robotics applications as datasets grow rapidly and tasks become more complex. We consider a probabilistic hierarchical modeling paradigm that combines the benefits of both worlds to deliver computationally efficient representations with inherent complexity regularization. We derive two efficient variational inference techniques to learn these representations and highlight the advantages of hierarchical infinite local regression models.
  arXiv  Detail & Related papers  (2022-11-02T13:54:07Z)
• HyperImpute: Generalized Iterative Imputation with Automatic Model Selection [77.86861638371926]
  We propose a generalized iterative imputation framework for adaptively and automatically configuring column-wise models. We provide a concrete implementation with out-of-the-box learners, simulators, and interfaces.
  arXiv  Detail & Related papers  (2022-06-15T19:10:35Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Principled inference of hyperedges and overlapping communities in hypergraphs [0.0]
  We propose a framework based on statistical inference to characterize the structural organization of hypergraphs. We show strong performance in hyperedge prediction tasks, detecting communities well aligned with the information carried by interactions, and robustness against addition of noisy hyperedges.
  arXiv  Detail & Related papers  (2022-04-12T09:13:46Z)
• Fast and More Powerful Selective Inference for Sparse High-order Interaction Model [17.549975092550074]
  We consider Sparse High-order Interaction Model (SHIM) in this study. Finding statistically significant high-order interactions is challenging due to intrinsic high dimensionality of the effects. Our main contribution is to extend the recently developed parametric programming approach for selective inference to high-order interaction models.
  arXiv  Detail & Related papers  (2021-06-09T09:22:42Z)
• Weighted Ensemble-model and Network Analysis: A method to predict fluid intelligence via naturalistic functional connectivity [2.66512000865131]
  We propose a new method namely Weighted Ensemble-model and Network Analysis. It combines the machine learning and graph theory for improved fluid intelligence prediction. Our proposed methods achieved best performance with 3.85 mean absolute deviation, 0.66 correlation coefficient and 0.42 R-squared coefficient.
  arXiv  Detail & Related papers  (2021-01-06T11:17:49Z)
• Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing [113.52575069030192]
  Big data, including applications with high security requirements, are often collected and stored on multiple heterogeneous devices, such as mobile devices, drones and vehicles. Due to the limitations of communication costs and security requirements, it is of paramount importance to extract information in a decentralized manner instead of aggregating data to a fusion center. We consider the problem of learning model parameters in a multi-agent system with data locally processed via distributed edge nodes. A class of mini-batch alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
  arXiv  Detail & Related papers  (2020-10-02T10:41:59Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)
• Exact representations of many body interactions with RBM neural networks [77.34726150561087]
  We exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators. This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as Pionless EFT in nuclear physics.
  arXiv  Detail & Related papers  (2020-05-07T15:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.