Unsupervised representation learning with recognition-parametrised probabilistic models

• URL: http://arxiv.org/abs/2209.05661v2
• Date: Thu, 20 Apr 2023 13:01:11 GMT
• Title: Unsupervised representation learning with recognition-parametrised probabilistic models
• Authors: William I.Walker, Hugo Soulat, Changmin Yu, Maneesh Sahani
• Abstract summary: We introduce a new approach to probabilistic unsupervised learning based on the recognition-parametrised model ( RPM) Under the key assumption that observations are conditionally independent given latents, the RPM combines parametric prior observation-conditioned latent distributions with non-parametric observationfactors. The RPM provides a powerful framework to discover meaningful latent structure underlying observational data, a function critical to both animal and artificial intelligence.
• Score: 12.865596223775649
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We introduce a new approach to probabilistic unsupervised learning based on the recognition-parametrised model (RPM): a normalised semi-parametric hypothesis class for joint distributions over observed and latent variables. Under the key assumption that observations are conditionally independent given latents, the RPM combines parametric prior and observation-conditioned latent distributions with non-parametric observation marginals. This approach leads to a flexible learnt recognition model capturing latent dependence between observations, without the need for an explicit, parametric generative model. The RPM admits exact maximum-likelihood learning for discrete latents, even for powerful neural-network-based recognition. We develop effective approximations applicable in the continuous-latent case. Experiments demonstrate the effectiveness of the RPM on high-dimensional data, learning image classification from weak indirect supervision; direct image-level latent Dirichlet allocation; and recognition-parametrised Gaussian process factor analysis (RP-GPFA) applied to multi-factorial spatiotemporal datasets. The RPM provides a powerful framework to discover meaningful latent structure underlying observational data, a function critical to both animal and artificial intelligence.

Related papers

• Semiparametric inference for impulse response functions using double/debiased machine learning [49.1574468325115]
  We introduce a machine learning estimator for the impulse response function (IRF) in settings where a time series of interest is subjected to multiple discrete treatments. The proposed estimator can rely on fully nonparametric relations between treatment and outcome variables, opening up the possibility to use flexible machine learning approaches to estimate IRFs.
  arXiv  Detail & Related papers  (2024-11-15T07:42:02Z)
• CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding [62.075029712357]
  This work introduces the Cognitive Diffusion Probabilistic Models (CogDPM) CogDPM features a precision estimation method based on the hierarchical sampling capabilities of diffusion models and weight the guidance with precision weights estimated by the inherent property of diffusion models. We apply CogDPM to real-world prediction tasks using the United Kindom precipitation and surface wind datasets.
  arXiv  Detail & Related papers  (2024-05-03T15:54:50Z)
• Prediction under Latent Subgroup Shifts with High-Dimensional Observations [30.433078066683848]
  We introduce a new approach to prediction in graphical models with latent-shift adaptation. Our novel form of RPM identifies causal latent structure in the source environment, and adapts properly to predict in the target.
  arXiv  Detail & Related papers  (2023-06-23T12:26:24Z)
• Biases in Inverse Ising Estimates of Near-Critical Behaviour [0.0]
  Inverse inference allows pairwise interactions to be reconstructed from empirical correlations. We show that estimators used for this inference, such as Pseudo-likelihood (PLM), are biased. Data-driven methods are explored and applied to a functional magnetic resonance imaging (fMRI) dataset from neuroscience.
  arXiv  Detail & Related papers  (2023-01-13T14:01:43Z)
• Spatio-temporally separable non-linear latent factor learning: an application to somatomotor cortex fMRI data [0.0]
  Models of fMRI data that can perform whole-brain discovery of latent factors are understudied. New methods for efficient spatial weight-sharing are critical to deal with the high dimensionality of the data and the presence of noise. Our approach is evaluated on data with multiple motor sub-tasks to assess whether the model captures disentangled latent factors that correspond to each sub-task.
  arXiv  Detail & Related papers  (2022-05-26T21:30:22Z)
• Masked prediction tasks: a parameter identifiability view [49.533046139235466]
  We focus on the widely used self-supervised learning method of predicting masked tokens. We show that there is a rich landscape of possibilities, out of which some prediction tasks yield identifiability, while others do not.
  arXiv  Detail & Related papers  (2022-02-18T17:09:32Z)
• Relational Neural Markov Random Fields [29.43155380361715]
  We introduce Markov Random Fields (RN-MRFs) which allow handling of complex hybrid domains. We propose a maximum pseudolikelihood estimation-based learning algorithm with importance for training the potential parameters.
  arXiv  Detail & Related papers  (2021-10-18T22:52:54Z)
• Counterfactual Maximum Likelihood Estimation for Training Deep Networks [83.44219640437657]
  Deep learning models are prone to learning spurious correlations that should not be learned as predictive clues. We propose a causality-based training framework to reduce the spurious correlations caused by observable confounders. We conduct experiments on two real-world tasks: Natural Language Inference (NLI) and Image Captioning.
  arXiv  Detail & Related papers  (2021-06-07T17:47:16Z)
• MINIMALIST: Mutual INformatIon Maximization for Amortized Likelihood Inference from Sampled Trajectories [61.3299263929289]
  Simulation-based inference enables learning the parameters of a model even when its likelihood cannot be computed in practice. One class of methods uses data simulated with different parameters to infer an amortized estimator for the likelihood-to-evidence ratio. We show that this approach can be formulated in terms of mutual information between model parameters and simulated data.
  arXiv  Detail & Related papers  (2021-06-03T12:59:16Z)
• Evaluating probabilistic classifiers: Reliability diagrams and score decompositions revisited [68.8204255655161]
  We introduce the CORP approach, which generates provably statistically Consistent, Optimally binned, and Reproducible reliability diagrams in an automated way. Corpor is based on non-parametric isotonic regression and implemented via the Pool-adjacent-violators (PAV) algorithm.
  arXiv  Detail & Related papers  (2020-08-07T08:22:26Z)
• Causal Modeling with Stochastic Confounders [11.881081802491183]
  This work extends causal inference with confounders. We propose a new approach to variational estimation for causal inference based on a representer theorem with a random input space.
  arXiv  Detail & Related papers  (2020-04-24T00:34:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.