Inferring Local Structure from Pairwise Correlations

• URL: http://arxiv.org/abs/2305.04386v2
• Date: Tue, 17 Oct 2023 22:51:42 GMT
• Title: Inferring Local Structure from Pairwise Correlations
• Authors: Mahajabin Rahman and Ilya Nemenman
• Abstract summary: We show that pairwise correlations provide enough information to recover local relations. This proves to be successful even though higher order interaction structures are present in our data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To construct models of large, multivariate complex systems, such as those in biology, one needs to constrain which variables are allowed to interact. This can be viewed as detecting "local" structures among the variables. In the context of a simple toy model of 2D natural and synthetic images, we show that pairwise correlations between the variables -- even when severely undersampled -- provide enough information to recover local relations, including the dimensionality of the data, and to reconstruct arrangement of pixels in fully scrambled images. This proves to be successful even though higher order interaction structures are present in our data. We build intuition behind the success, which we hope might contribute to modeling complex, multivariate systems and to explaining the success of modern attention-based machine learning approaches.

Related papers

• How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model [4.215221129670858]
  We show that by introducing sparsity to generative hierarchical models of data, the task acquires insensitivity to spatial transformations that are discrete versions of smooth transformations. We quantify how the sample complexity of CNNs learning the SRHM depends on both the sparsity and hierarchical structure of the task.
  arXiv  Detail & Related papers  (2024-04-16T17:01:27Z)
• Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration [50.62725807357586]
  This article presents a general Bayesian learning framework for multi-modal groupwise image registration. We propose a novel hierarchical variational auto-encoding architecture to realise the inference procedure of the latent variables. Experiments were conducted to validate the proposed framework, including four different datasets from cardiac, brain, and abdominal medical images.
  arXiv  Detail & Related papers  (2024-01-04T08:46:39Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Dynamic Latent Separation for Deep Learning [67.62190501599176]
  A core problem in machine learning is to learn expressive latent variables for model prediction on complex data. Here, we develop an approach that improves expressiveness, provides partial interpretation, and is not restricted to specific applications.
  arXiv  Detail & Related papers  (2022-10-07T17:56:53Z)
• Unsupervised Multimodal Change Detection Based on Structural Relationship Graph Representation Learning [40.631724905575034]
  Unsupervised multimodal change detection is a practical and challenging topic that can play an important role in time-sensitive emergency applications. We take advantage of two types of modality-independent structural relationships in multimodal images. We present a structural relationship graph representation learning framework for measuring the similarity of the two structural relationships.
  arXiv  Detail & Related papers  (2022-10-03T13:55:08Z)
• Amortised Inference in Structured Generative Models with Explaining Away [16.92791301062903]
  We extend the output of amortised variational inference to incorporate structured factors over multiple variables. We show that appropriately parameterised factors can be combined efficiently with variational message passing in elaborate graphical structures. We then fit the structured model to high-dimensional neural spiking time-series from the hippocampus of freely moving rodents.
  arXiv  Detail & Related papers  (2022-09-12T12:52:15Z)
• Amortized Inference for Causal Structure Learning [72.84105256353801]
  Learning causal structure poses a search problem that typically involves evaluating structures using a score or independence test. We train a variational inference model to predict the causal structure from observational/interventional data. Our models exhibit robust generalization capabilities under substantial distribution shift.
  arXiv  Detail & Related papers  (2022-05-25T17:37:08Z)
• Joint Gaussian Graphical Model Estimation: A Survey [31.811209829224293]
  Graphs from complex systems often share a partial underlying structure across domains while retaining individual features. Growing evidence shows that the shared structure across domains boosts the estimation power of graphs. This manuscript surveys recent work on statistical inference of joint Gaussian graphical models.
  arXiv  Detail & Related papers  (2021-10-19T21:56:27Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)
• A Critical View of the Structural Causal Model [89.43277111586258]
  We show that one can identify the cause and the effect without considering their interaction at all. We propose a new adversarial training method that mimics the disentangled structure of the causal model. Our multidimensional method outperforms the literature methods on both synthetic and real world datasets.
  arXiv  Detail & Related papers  (2020-02-23T22:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.