The Degeneracy Distillery

• URL: http://arxiv.org/abs/2606.23838v1
• Date: Mon, 22 Jun 2026 18:18:09 GMT
• Title: The Degeneracy Distillery
• Abstract summary: We present the degeneracy distillery, a method that detects and resolves degenerate parameter combinations.<n>By exploring the information geometry of the likelihood, we characterize degeneracies as an intrinsic property of the physical model.<n>We demonstrate our approach on a range of synthetic and real-world problems, discovering symbolic coordinate transformations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When two or more parameters or labels produce similar data, they are degenerate, or hard to distinguish. Degeneracies render both label prediction and inverse problems difficult, since both machine learning algorithms and probabilistic samplers rely on the distinguishability of data and its gradients with respect to parameters. However, identifying degeneracies in physical models or real-world datasets can be elucidating about the choice of model or the underlying process that produces the data. We present the degeneracy distillery, a method that (1) detects and (2) resolves degenerate parameter combinations (a) automatically and (b) symbolically, from parameter-data (or parameter-simulation) pairs alone, through estimation and flattening of the Fisher information matrix. By exploring the information geometry of the likelihood, we characterize degeneracies as an intrinsic property of the physical model, requiring no realised data observation. We demonstrate our approach on a range of synthetic and real-world problems, discovering symbolic coordinate transformations that identify the combinations of parameters of a model which yield independent effects on the data. The resulting coordinates flatten the Fisher information in expectation globally, in contrast to posterior-based methods that flatten only at a single point, and substantially reduce the simulation budget required for downstream neural posterior estimation. In test cases we require up to $10\times$ fewer simulations for posterior estimation at matched validation calibration whilst simultaneously gaining physical insight on the system.

Related papers

• Machine Learning Surrogate Modeling for Homogenization of Hyperelastic Materials with Boolean Microstructures [0.0]
  Data-driven surrogate models are an alternative to numerical homogenization of heterogeneous materials.<n>A neural network is trained on combinations of scalar and curve-wise statistical descriptors.<n>Additional data representing limiting cases of the parameter space are incorporated to stabilize training and improve extrapolation behavior.
  arXiv  Detail & Related papers  (2026-05-31T00:51:52Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Efficient Deconvolution in Populational Inverse Problems [6.1048542188987085]
  This work is focussed on the inversion task of inferring the distribution over parameters of interest leading to multiple sets of observations.<n>The potential to solve such distributional inversion problems is driven by increasing availability of data, but a major roadblock is blind deconvolution.<n>We propose a methodology leveraging large data sets of observations, collected from different instantiations of the same physical processes, to simultaneously deconvolve the data corrupting noise distribution.
  arXiv  Detail & Related papers  (2025-05-26T11:25:46Z)
• An Iterative Bayesian Approach for System Identification based on Linear Gaussian Models [70.75865595489691]
  We tackle the problem of system identification, where we select inputs, observe the corresponding outputs from the true system, and optimize the parameters of our model to best fit the data.<n>Our approach only requires input-output data from the system and first-order information of the model with respect to the parameters.
  arXiv  Detail & Related papers  (2025-01-28T01:57:51Z)
• Bayesian Surrogate Training on Multiple Data Sources: A Hybrid Modeling Strategy [3.2926386146887796]
  We propose two novel approaches to integrate simulation data and real-world measurement data during surrogate training.<n>The first method trains separate surrogate models for each data source and combines their predictive distributions, while the second incorporates both data sources by training a single surrogate.
  arXiv  Detail & Related papers  (2024-12-16T15:27:28Z)
• Beyond Convergence: Identifiability of Machine Learning and Deep Learning Models [0.0]
  We investigate the notion of model parameter identifiability through a case study focused on parameter estimation from motion sensor data. We employ a deep neural network to estimate subject-wise parameters, including mass, stiffness, and equilibrium leg length. The results show that while certain parameters can be identified from the observation data, others remain unidentifiable.
  arXiv  Detail & Related papers  (2023-07-21T03:40:53Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Learning to Bound Counterfactual Inference in Structural Causal Models from Observational and Randomised Data [64.96984404868411]
  We derive a likelihood characterisation for the overall data that leads us to extend a previous EM-based algorithm. The new algorithm learns to approximate the (unidentifiability) region of model parameters from such mixed data sources. It delivers interval approximations to counterfactual results, which collapse to points in the identifiable case.
  arXiv  Detail & Related papers  (2022-12-06T12:42:11Z)
• A Causality-Based Learning Approach for Discovering the Underlying Dynamics of Complex Systems from Partial Observations with Stochastic Parameterization [1.2882319878552302]
  This paper develops a new iterative learning algorithm for complex turbulent systems with partial observations. It alternates between identifying model structures, recovering unobserved variables, and estimating parameters. Numerical experiments show that the new algorithm succeeds in identifying the model structure and providing suitable parameterizations for many complex nonlinear systems.
  arXiv  Detail & Related papers  (2022-08-19T00:35:03Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Asymptotic Analysis of an Ensemble of Randomly Projected Linear Discriminants [94.46276668068327]
  In [1], an ensemble of randomly projected linear discriminants is used to classify datasets. We develop a consistent estimator of the misclassification probability as an alternative to the computationally-costly cross-validation estimator. We also demonstrate the use of our estimator for tuning the projection dimension on both real and synthetic data.
  arXiv  Detail & Related papers  (2020-04-17T12:47:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.