AutoBayes: A Compositional Framework for Generalized Variational Inference

• URL: http://arxiv.org/abs/2503.18608v2
• Date: Tue, 25 Mar 2025 10:55:49 GMT
• Title: AutoBayes: A Compositional Framework for Generalized Variational Inference
• Authors: Toby St Clere Smithe, Marco Perin,
• Abstract summary: We introduce a new compositional framework for generalized variational inference.<n>We explain that exact Bayesian inference and the loss functions typical of variational inference satisfy chain rules akin to that of reverse-mode automatic differentiation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a new compositional framework for generalized variational inference, clarifying the different parts of a model, how they interact, and how they compose. We explain that both exact Bayesian inference and the loss functions typical of variational inference (such as variational free energy and its generalizations) satisfy chain rules akin to that of reverse-mode automatic differentiation, and we advocate for exploiting this to build and optimize models accordingly. To this end, we construct a series of compositional tools: for building models; for constructing their inversions; for attaching local loss functions; and for exposing parameters. Finally, we explain how the resulting parameterized statistical games may be optimized locally, too. We illustrate our framework with a number of classic examples, pointing to new areas of extensibility that are revealed.

Related papers

• Amortized In-Context Bayesian Posterior Estimation [15.714462115687096]
  Amortization, through conditional estimation, is a viable strategy to alleviate such difficulties.<n>We conduct a thorough comparative analysis of amortized in-context Bayesian posterior estimation methods.<n>We highlight the superiority of the reverse KL estimator for predictive problems, especially when combined with the transformer architecture and normalizing flows.
  arXiv  Detail & Related papers  (2025-02-10T16:00:48Z)
• A Fixed-Point Approach for Causal Generative Modeling [20.88890689294816]
  We propose a novel formalism for describing Structural Causal Models (SCMs) as fixed-point problems on causally ordered variables.<n>We establish the weakest known conditions for their unique recovery given the topological ordering (TO)
  arXiv  Detail & Related papers  (2024-04-10T12:29:05Z)
• Nonparametric Partial Disentanglement via Mechanism Sparsity: Sparse Actions, Interventions and Sparse Temporal Dependencies [58.179981892921056]
  This work introduces a novel principle for disentanglement we call mechanism sparsity regularization. We propose a representation learning method that induces disentanglement by simultaneously learning the latent factors. We show that the latent factors can be recovered by regularizing the learned causal graph to be sparse.
  arXiv  Detail & Related papers  (2024-01-10T02:38:21Z)
• Flow Factorized Representation Learning [109.51947536586677]
  We introduce a generative model which specifies a distinct set of latent probability paths that define different input transformations. We show that our model achieves higher likelihoods on standard representation learning benchmarks while simultaneously being closer to approximately equivariant models.
  arXiv  Detail & Related papers  (2023-09-22T20:15:37Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Recursive Monte Carlo and Variational Inference with Auxiliary Variables [64.25762042361839]
  Recursive auxiliary-variable inference (RAVI) is a new framework for exploiting flexible proposals. RAVI generalizes and unifies several existing methods for inference with expressive expressive families. We show RAVI's design framework and theorems by using them to analyze and improve upon Salimans et al.'s Markov Chain Variational Inference.
  arXiv  Detail & Related papers  (2022-03-05T23:52:40Z)
• Towards Robust and Adaptive Motion Forecasting: A Causal Representation Perspective [72.55093886515824]
  We introduce a causal formalism of motion forecasting, which casts the problem as a dynamic process with three groups of latent variables. We devise a modular architecture that factorizes the representations of invariant mechanisms and style confounders to approximate a causal graph. Experiment results on synthetic and real datasets show that our three proposed components significantly improve the robustness and reusability of the learned motion representations.
  arXiv  Detail & Related papers  (2021-11-29T18:59:09Z)
• Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Set [50.67431815647126]
  Post-hoc global/local feature attribution methods are being progressively employed to understand machine learning models. We show that partial orders of local/global feature importance arise from this methodology. We show that every relation among features present in these partial orders also holds in the rankings provided by existing approaches.
  arXiv  Detail & Related papers  (2021-10-26T02:53:14Z)
• CARE: Coherent Actionable Recourse based on Sound Counterfactual Explanations [0.0]
  This paper introduces CARE, a modular explanation framework that addresses the model- and user-level desiderata. As a model-agnostic approach, CARE generates multiple, diverse explanations for any black-box model.
  arXiv  Detail & Related papers  (2021-08-18T15:26:59Z)
• Compositional Abstraction Error and a Category of Causal Models [2.291640606078406]
  We argue that compositionality is a desideratum for model transformations and the associated errors. We develop a framework for model transformations and abstractions with a notion of error that is compositional.
  arXiv  Detail & Related papers  (2021-03-29T16:48:12Z)
• A Variational View on Bootstrap Ensembles as Bayesian Inference [24.55506395666038]
  We consider an ensemble-based scheme where each model/particle corresponds to a perturbation of the data by means of parametric bootstrap and a perturbation of the prior. Experiments confirm that ensemble methods can be a valid alternative to approximate Bayesian inference.
  arXiv  Detail & Related papers  (2020-06-08T13:01:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.