Don't Stop Me Yet: Sampling Loss Minima via Dissipative Riemannian Mechanics

• URL: http://arxiv.org/abs/2605.15459v1
• Date: Thu, 14 May 2026 22:41:01 GMT
• Title: Don't Stop Me Yet: Sampling Loss Minima via Dissipative Riemannian Mechanics
• Abstract summary: We propose a sampler for characterizing minima of neural network loss functions.<n>Our proposed sampler, DiMS, is guaranteed to sample exactly from the minimum level sets.<n>We consider uncertainty in Bayesian inference as the motivating problem and observe improved performance compared to previously proposed approaches.
• Score: 3.567534554383171
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The minima of modern neural network loss functions are typically not isolated, rather they form connected components of reparameterization invariant solutions on the training data. Analytically characterizing these solutions is a hard problem, but sampling approaches are feasible. By construction, existing methods either spread over low-loss regions, and thus do not sample reparameterization invariant solutions exactly, or are inherently local, which limits exploration of other minima valleys. We propose sampling such reparameterization invariant models using a dynamical system based on kinetic energy, subject to a gravitational pull and a friction term that dissipates energy from the system. Our proposed sampler, DiMS, is guaranteed to sample exactly from the minimum level sets and depends on physically motivated hyperparameters which allows control over the exploration capabilities of the sampler. We consider uncertainty quantification in Bayesian inference as the motivating problem and observe improved performance compared to previously proposed approaches.

Related papers

• Neural Network Quantization by Learning Low-Loss Subspaces [42.07461306663832]
  Recent studies indicate that low-loss FP solutions are not isolated, but instead belong to connected low-loss subspaces of the loss landscape.<n>This raises the question: can a quantized model be constructed to lie within a low-loss subspace of the FP model, thereby automatically preserving performance?<n>We address this question by learning quantization-aware linear paths in weight space optimized to minimize loss.<n>We demonstrate that the midpoint of the resulting subspace is, by design, quantization-friendly and that its direct quantization yields performance comparable to that of quantization-aware training.
  arXiv  Detail & Related papers  (2026-06-23T18:48:45Z)
• Residual-loss Anomaly Analysis of Physics-Informed Neural Networks: An Inverse Method for Change-point Detection in Nonlinear Dynamical Systems with Regime Switching [10.645192584041162]
  We propose a unified framework for residual-loss anomaly analysis of physics-informed neural networks.<n>This approach jointly infers piecewise parameters and transition points under a single set of constraints.<n>We show that the proposed method outperforms traditional decoupled approaches in both change-point localization and parameter estimation accuracy.
  arXiv  Detail & Related papers  (2026-04-28T13:50:50Z)
• Variance-Reduced Model Predictive Path Integral via Quadratic Model Approximation [18.217598791860684]
  We introduce a hybrid variance-reduced MPPI framework that integrates a prior model into the sampling process.<n>We demonstrate that adopting a quadratic approximation enables the derivation of a closed-form, model-guided prior that effectively concentrates samples in informative regions.
  arXiv  Detail & Related papers  (2026-02-03T15:23:17Z)
• Generative diffusion for perceptron problems: statistical physics analysis and efficient algorithms [2.860608352191896]
  We consider random instances of non- numerically weights perceptron problems in the high-dimensional limit.<n>We develop a formalism based on replica theory to predict Approximate sampling space using generative algorithms.
  arXiv  Detail & Related papers  (2025-02-22T16:43:01Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• Amortized Posterior Sampling with Diffusion Prior Distillation [55.03585818289934]
  Amortized Posterior Sampling is a novel variational inference approach for efficient posterior sampling in inverse problems.<n>Our method trains a conditional flow model to minimize the divergence between the variational distribution and the posterior distribution implicitly defined by the diffusion model.<n>Unlike existing methods, our approach is unsupervised, requires no paired training data, and is applicable to both Euclidean and non-Euclidean domains.
  arXiv  Detail & Related papers  (2024-07-25T09:53:12Z)
• Dynamical Measure Transport and Neural PDE Solvers for Sampling [77.38204731939273]
  We tackle the task of sampling from a probability density as transporting a tractable density function to the target. We employ physics-informed neural networks (PINNs) to approximate the respective partial differential equations (PDEs) solutions. PINNs allow for simulation- and discretization-free optimization and can be trained very efficiently.
  arXiv  Detail & Related papers  (2024-07-10T17:39:50Z)
• Dynamical Hyperspectral Unmixing with Variational Recurrent Neural Networks [25.051918587650636]
  Multitemporal hyperspectral unmixing (MTHU) is a fundamental tool in the analysis of hyperspectral image sequences. We propose an unsupervised MTHU algorithm based on variational recurrent neural networks.
  arXiv  Detail & Related papers  (2023-03-19T04:51:34Z)
• Uncovering Challenges of Solving the Continuous Gromov-Wasserstein Problem [68.07116119373565]
  The Gromov-Wasserstein Optimal Transport (GWOT) problem has attracted the special attention of the ML community.<n>We crash-test existing continuous GWOT approaches on different scenarios, carefully record and analyze the obtained results, and identify issues.<n>We propose a new continuous GWOT method which does not rely on discrete techniques and partially solves some of the problems of the competitors.
  arXiv  Detail & Related papers  (2023-03-10T15:21:12Z)
• History-Based, Bayesian, Closure for Stochastic Parameterization: Application to Lorenz '96 [0.09137554315375918]
  We develop a new type of parameterization based on a Bayesian formalism for neural networks, to account for uncertainty quantification. We apply the proposed Bayesian history-based parameterization to the Lorenz '96 model in the presence of noisy and sparse data. This approach paves the way for the use of Bayesian approaches for closure problems.
  arXiv  Detail & Related papers  (2022-10-26T05:22:50Z)
• Sampling with Mollified Interaction Energy Descent [57.00583139477843]
  We present a new optimization-based method for sampling called mollified interaction energy descent (MIED) MIED minimizes a new class of energies on probability measures called mollified interaction energies (MIEs) We show experimentally that for unconstrained sampling problems our algorithm performs on par with existing particle-based algorithms like SVGD.
  arXiv  Detail & Related papers  (2022-10-24T16:54:18Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise [51.080620762639434]
  We propose a more efficient parameterization of the posterior approximation for sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks.
  arXiv  Detail & Related papers  (2021-03-15T16:16:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.