Stein's method for marginals on large graphical models

• URL: http://arxiv.org/abs/2410.11771v2
• Date: Wed, 07 May 2025 13:51:47 GMT
• Title: Stein's method for marginals on large graphical models
• Authors: Tiangang Cui, Shuigen Liu, Xin T. Tong,
• Abstract summary: We introduce a novel $delta$-locality condition that quantifies the locality in distributions.<n>We show that these methods greatly reduce the sample complexity and computational cost via localized and parallel implementations.
• Score: 1.8843687952462742
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many spatial models exhibit locality structures that effectively reduce their intrinsic dimensionality, enabling efficient approximation and sampling of high-dimensional distributions. However, existing approximation techniques mainly focus on joint distributions, and do not guarantee accuracy for low-dimensional marginals. By leveraging the locality structures, we establish a dimension independent uniform error bound for the marginals of approximate distributions. Inspired by the Stein's method, we introduce a novel $\delta$-locality condition that quantifies the locality in distributions, and link it to the structural assumptions such as the sparse graphical models. The theoretical guarantee motivates the localization of existing sampling methods, as we illustrate through the localized likelihood-informed subspace method and localized score matching. We show that by leveraging the locality structure, these methods greatly reduce the sample complexity and computational cost via localized and parallel implementations.

Related papers

• Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling [62.640128548633946]
  We introduce a novel inference-time scaling approach based on particle Gibbs sampling for discrete diffusion models.<n>Our method consistently outperforms prior inference-time strategies on reward-guided text generation tasks.
  arXiv  Detail & Related papers  (2025-07-11T08:00:47Z)
• Joint stochastic localization and applications [7.22482471589943]
  We introduce a joint localization framework for constructing couplings between probability distributions.<n>We also introduce a family of distributional distances based on the couplings induced by joint localization.<n>The proposed distances also motivate new methods for distribution estimation that are of independent interest.
  arXiv  Detail & Related papers  (2025-05-19T17:47:05Z)
• Localized Diffusion Models for High Dimensional Distributions Generation [1.677718351174347]
  We study the importance of better understanding and exploiting low-dimensional structure in the target distribution.<n>Under locality structure, the score function is effectively low-dimensional, so that it can be estimated by a localized neural network with significantly reduced sample complexity.<n>We prove that such localization enables diffusion models to circumvent the curse of dimensionality (CoD)
  arXiv  Detail & Related papers  (2025-05-07T13:51:50Z)
• Adaptive $k$-nearest neighbor classifier based on the local estimation of the shape operator [49.87315310656657]
  We introduce a new adaptive $k$-nearest neighbours ($kK$-NN) algorithm that explores the local curvature at a sample to adaptively defining the neighborhood size. Results on many real-world datasets indicate that the new $kK$-NN algorithm yields superior balanced accuracy compared to the established $k$-NN method.
  arXiv  Detail & Related papers  (2024-09-08T13:08:45Z)
• MASALA: Model-Agnostic Surrogate Explanations by Locality Adaptation [3.587367153279351]
  Existing local Explainable AI (XAI) methods select a region of the input space in the vicinity of a given input instance, for which they approximate the behaviour of a model using a simpler and more interpretable surrogate model. We propose a novel method, MASALA, for generating explanations, which automatically determines the appropriate local region of impactful model behaviour for each individual instance being explained.
  arXiv  Detail & Related papers  (2024-08-19T15:26:45Z)
• Rejection via Learning Density Ratios [50.91522897152437]
  Classification with rejection emerges as a learning paradigm which allows models to abstain from making predictions. We propose a different distributional perspective, where we seek to find an idealized data distribution which maximizes a pretrained model's performance. Our framework is tested empirically over clean and noisy datasets.
  arXiv  Detail & Related papers  (2024-05-29T01:32:17Z)
• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models [6.76974373198208]
  We find that the dependency of the error incurred within each denoising step on the ambient dimension $d$ is in general unavoidable.<n>This represents the first theoretical demonstration that the DDPM sampler can adapt to unknown low-dimensional structures in the target distribution.
  arXiv  Detail & Related papers  (2024-05-23T17:59:10Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Guarantee Regions for Local Explanations [29.429229877959663]
  We propose an anchor-based algorithm for identifying regions in which local explanations are guaranteed to be correct. Our method produces an interpretable feature-aligned box where the prediction of the local surrogate model is guaranteed to match the predictive model.
  arXiv  Detail & Related papers  (2024-02-20T06:04:44Z)
• Distributed Markov Chain Monte Carlo Sampling based on the Alternating Direction Method of Multipliers [143.6249073384419]
  In this paper, we propose a distributed sampling scheme based on the alternating direction method of multipliers. We provide both theoretical guarantees of our algorithm's convergence and experimental evidence of its superiority to the state-of-the-art. In simulation, we deploy our algorithm on linear and logistic regression tasks and illustrate its fast convergence compared to existing gradient-based methods.
  arXiv  Detail & Related papers  (2024-01-29T02:08:40Z)
• Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data [68.62134204367668]
  This paper studies score approximation, estimation, and distribution recovery of diffusion models, when data are supported on an unknown low-dimensional linear subspace. We show that with a properly chosen neural network architecture, the score function can be both accurately approximated and efficiently estimated. The generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
  arXiv  Detail & Related papers  (2023-02-14T17:02:35Z)
• Optimal Scaling for Locally Balanced Proposals in Discrete Spaces [65.14092237705476]
  We show that efficiency of Metropolis-Hastings (M-H) algorithms in discrete spaces can be characterized by an acceptance rate that is independent of the target distribution. Knowledge of the optimal acceptance rate allows one to automatically tune the neighborhood size of a proposal distribution in a discrete space, directly analogous to step-size control in continuous spaces.
  arXiv  Detail & Related papers  (2022-09-16T22:09:53Z)
• Density Ratio Estimation via Infinitesimal Classification [85.08255198145304]
  We propose DRE-infty, a divide-and-conquer approach to reduce Density ratio estimation (DRE) to a series of easier subproblems. Inspired by Monte Carlo methods, we smoothly interpolate between the two distributions via an infinite continuum of intermediate bridge distributions. We show that our approach performs well on downstream tasks such as mutual information estimation and energy-based modeling on complex, high-dimensional datasets.
  arXiv  Detail & Related papers  (2021-11-22T06:26:29Z)
• A likelihood approach to nonparametric estimation of a singular distribution using deep generative models [4.329951775163721]
  We investigate a likelihood approach to nonparametric estimation of a singular distribution using deep generative models. We prove that a novel and effective solution exists by perturbing the data with an instance noise. We also characterize the class of distributions that can be efficiently estimated via deep generative models.
  arXiv  Detail & Related papers  (2021-05-09T23:13:58Z)
• Arbitrary Conditional Distributions with Energy [11.081460215563633]
  A more general and useful problem is arbitrary conditional density estimation. We propose a novel method, Arbitrary Conditioning with Energy (ACE), that can simultaneously estimate the distribution $p(mathbfx_u mid mathbfx_o)$. We also simplify the learning problem by only learning one-dimensional conditionals, from which more complex distributions can be recovered during inference.
  arXiv  Detail & Related papers  (2021-02-08T18:36:26Z)
• Linear Optimal Transport Embedding: Provable Wasserstein classification for certain rigid transformations and perturbations [79.23797234241471]
  Discriminating between distributions is an important problem in a number of scientific fields. The Linear Optimal Transportation (LOT) embeds the space of distributions into an $L2$-space. We demonstrate the benefits of LOT on a number of distribution classification problems.
  arXiv  Detail & Related papers  (2020-08-20T19:09:33Z)
• Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization [101.5159744660701]
  In distributed second order optimization, a standard strategy is to average many local estimates, each of which is based on a small sketch or batch of the data. Here, we introduce a new technique for debiasing the local estimates, which leads to both theoretical and empirical improvements in the convergence rate of distributed second order methods.
  arXiv  Detail & Related papers  (2020-07-02T18:08:14Z)
• Log-Likelihood Ratio Minimizing Flows: Towards Robust and Quantifiable Neural Distribution Alignment [52.02794488304448]
  We propose a new distribution alignment method based on a log-likelihood ratio statistic and normalizing flows. We experimentally verify that minimizing the resulting objective results in domain alignment that preserves the local structure of input domains.
  arXiv  Detail & Related papers  (2020-03-26T22:10:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.