Distributional Sensitivity Analysis: Enabling Differentiability in Sample-Based Inference

• URL: http://arxiv.org/abs/2508.09347v1
• Date: Tue, 12 Aug 2025 21:21:45 GMT
• Title: Distributional Sensitivity Analysis: Enabling Differentiability in Sample-Based Inference
• Authors: Pi-Yueh Chuang, Ahmed Attia, Emil Constantinescu,
• Abstract summary: We present two analytical formulae for estimating the sensitivity at given realizations of an arbitrary-dimensional random vector.<n>The first formula interprets this sensitivity as partial derivatives of the inverse mapping associated with the vector of 1-D conditional distributions.<n>The second formula introduces a diagonal approximation that reduces computational cost at the cost of some accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present two analytical formulae for estimating the sensitivity -- namely, the gradient or Jacobian -- at given realizations of an arbitrary-dimensional random vector with respect to its distributional parameters. The first formula interprets this sensitivity as partial derivatives of the inverse mapping associated with the vector of 1-D conditional distributions. The second formula, intended for optimization methods that tolerate inexact gradients, introduces a diagonal approximation that reduces computational cost at the cost of some accuracy. We additionally provide four second-order numerical algorithms to approximate both formulae when closed forms are unavailable. We performed verification and validation studies to demonstrate the correctness of these numerical algorithms and the effectiveness of the proposed formulae. A nuclear physics application showcases how our work enables uncertainty quantification and parameter inference for quantum correlation functions. Our approach differs from existing methods by avoiding the need for model fitting, knowledge of sampling algorithms, and evaluation of high-dimensional integrals. It is therefore particularly useful for sample-based inverse problems when the sampler operates as a black box or requires expensive physics simulations. Moreover, our method renders arbitrary sampling subroutines differentiable, facilitating their integration into programming frameworks for deep learning and automatic differentiation. Algorithmic details and code implementations are provided in this paper and in our open-source software DistroSA to enable reproducibility and further development.

Related papers

• Enhanced Derivative-Free Optimization Using Adaptive Correlation-Induced Finite Difference Estimators [6.054123928890574]
  We develop an algorithm designed to enhance DFO in terms of both gradient estimation efficiency and sample efficiency.<n>We establish the consistency of our proposed algorithm and demonstrate that, despite using a batch of samples per iteration, it achieves the same convergence rate as the KW and SPSA methods.
  arXiv  Detail & Related papers  (2025-02-28T08:05:54Z)
• Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models [57.52124921268249]
  We propose a Trust Sequential Quadratic Programming method to find both first and second-order stationary points. To converge to first-order stationary points, our method computes a gradient step in each iteration defined by minimizing a approximation of the objective subject. To converge to second-order stationary points, our method additionally computes an eigen step to explore the negative curvature the reduced Hessian matrix.
  arXiv  Detail & Related papers  (2024-09-24T04:39:47Z)
• Harmonic Path Integral Diffusion [0.4527270266697462]
  We present a novel approach for sampling from a continuous multivariate probability distribution, which may either be explicitly known (up to a normalization factor) or represented via empirical samples. Our method constructs a time-dependent bridge from a delta function centered at the origin of the state space at $t=0$, transforming it into the target distribution at $t=1$. We contrast these algorithms with other sampling methods, particularly simulated and path integral sampling, highlighting their advantages in terms of analytical control, accuracy, and computational efficiency.
  arXiv  Detail & Related papers  (2024-09-23T16:20:21Z)
• Shape-informed surrogate models based on signed distance function domain encoding [8.052704959617207]
  We propose a non-intrusive method to build surrogate models that approximate the solution of parameterized partial differential equations (PDEs) Our approach is based on the combination of two neural networks (NNs)
  arXiv  Detail & Related papers  (2024-09-19T01:47:04Z)
• Stochastic Optimization for Non-convex Problem with Inexact Hessian Matrix, Gradient, and Function [99.31457740916815]
  Trust-region (TR) and adaptive regularization using cubics have proven to have some very appealing theoretical properties. We show that TR and ARC methods can simultaneously provide inexact computations of the Hessian, gradient, and function values.
  arXiv  Detail & Related papers  (2023-10-18T10:29:58Z)
• Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
  Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
  arXiv  Detail & Related papers  (2023-06-13T01:18:16Z)
• VI-DGP: A variational inference method with deep generative prior for solving high-dimensional inverse problems [0.7734726150561089]
  We propose a novel approximation method for estimating the high-dimensional posterior distribution. This approach leverages a deep generative model to learn a prior model capable of generating spatially-varying parameters. The proposed method can be fully implemented in an automatic differentiation manner.
  arXiv  Detail & Related papers  (2023-02-22T06:48:10Z)
• Fast Computation of Optimal Transport via Entropy-Regularized Extragradient Methods [75.34939761152587]
  Efficient computation of the optimal transport distance between two distributions serves as an algorithm that empowers various applications. This paper develops a scalable first-order optimization-based method that computes optimal transport to within $varepsilon$ additive accuracy.
  arXiv  Detail & Related papers  (2023-01-30T15:46:39Z)
• Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models [0.0]
  We introduce a manifold learning-based method for uncertainty quantification (UQ) in describing systems. The proposed method is able to achieve highly accurate approximations which ultimately lead to the significant acceleration of UQ tasks.
  arXiv  Detail & Related papers  (2021-07-21T00:24:15Z)
• Online Statistical Inference for Stochastic Optimization via Kiefer-Wolfowitz Methods [8.890430804063705]
  We first present the distribution for the Polyak-Ruppert-averaging type Kiefer-Wolfowitz (AKW) estimators. The distributional result reflects the trade-off between statistical efficiency and function query complexity.
  arXiv  Detail & Related papers  (2021-02-05T19:22:41Z)
• Zeroth-Order Hybrid Gradient Descent: Towards A Principled Black-Box Optimization Framework [100.36569795440889]
  This work is on the iteration of zero-th-order (ZO) optimization which does not require first-order information. We show that with a graceful design in coordinate importance sampling, the proposed ZO optimization method is efficient both in terms of complexity as well as as function query cost.
  arXiv  Detail & Related papers  (2020-12-21T17:29:58Z)
• Cogradient Descent for Bilinear Optimization [124.45816011848096]
  We introduce a Cogradient Descent algorithm (CoGD) to address the bilinear problem. We solve one variable by considering its coupling relationship with the other, leading to a synchronous gradient descent. Our algorithm is applied to solve problems with one variable under the sparsity constraint.
  arXiv  Detail & Related papers  (2020-06-16T13:41:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.