Evidence Slopes and Effective Dimension in Singular Linear Models

• URL: http://arxiv.org/abs/2601.01238v1
• Date: Sat, 03 Jan 2026 17:05:55 GMT
• Title: Evidence Slopes and Effective Dimension in Singular Linear Models
• Abstract summary: Singular learning theory replaces Laplace approximation with the real canonical threshold (RLCT)<n>We show theoretically and empirically that the error of Laplace/BIC times log n grows linearly with (d/2 minus) times log n.<n>Our results provide a concrete finite-sample characterization of Laplace failure in singular models and demonstrate that evidence slopes can be used as a practical estimator of effective dimension in simple linear settings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian model selection commonly relies on Laplace approximation or the Bayesian Information Criterion (BIC), which assume that the effective model dimension equals the number of parameters. Singular learning theory replaces this assumption with the real log canonical threshold (RLCT), an effective dimension that can be strictly smaller in overparameterized or rank-deficient models. We study linear-Gaussian rank models and linear subspace (dictionary) models in which the exact marginal likelihood is available in closed form and the RLCT is analytically tractable. In this setting, we show theoretically and empirically that the error of Laplace/BIC grows linearly with (d/2 minus lambda) times log n, where d is the ambient parameter dimension and lambda is the RLCT. An RLCT-aware correction recovers the correct evidence slope and is invariant to overcomplete reparameterizations that represent the same data subspace. Our results provide a concrete finite-sample characterization of Laplace failure in singular models and demonstrate that evidence slopes can be used as a practical estimator of effective dimension in simple linear settings.

Related papers

• Tubular Riemannian Laplace Approximations for Bayesian Neural Networks [0.0]
  Laplace approximations are among the simplest and most practical methods for approximate Bayesian inference in neural networks.<n>Recent work has proposed geometric Gaussian approximations to adapt to this structure.<n>We introduce the Tubular Riemannian Laplace (TRL) approximation.
  arXiv  Detail & Related papers  (2025-12-30T17:50:55Z)
• When does Gaussian equivalence fail and how to fix it: Non-universal behavior of random features with quadratic scaling [15.148577493784051]
  Gaussian equivalence theory (GET) states that the behavior of high-dimensional, complex features can be captured by Gaussian surrogates.<n>But numerical experiments show that this equivalence can fail even for simple embeddings under general scaling regimes.<n>We introduce a Conditional Equivalent (CGE) model, which can be viewed as appending a low-dimensional non-Gaussian component to an otherwise high-dimensional Gaussian model.
  arXiv  Detail & Related papers  (2025-12-03T00:23:12Z)
• Efficient Covariance Estimation for Sparsified Functional Data [51.69796254617083]
  proposed Random-knots (Random-knots-Spatial) and B-spline (Bspline-Spatial) estimators of the covariance function are computationally efficient.<n>Asymptotic pointwise of the covariance are obtained for sparsified individual trajectories under some regularity conditions.
  arXiv  Detail & Related papers  (2025-11-23T00:50:33Z)
• Deep Partially Linear Transformation Model for Right-Censored Survival Data [6.315323176162257]
  This paper introduces a deep partially linear transformation model (DPLTM) as a general and flexible regression framework.<n>The proposed method is capable of avoiding the curse of dimensionality while still retaining the interpretability of some cocensors of interest.<n> Comprehensive simulation studies demonstrate the impressive performance of the proposed procedure in terms of both accuracy and the predictive power.
  arXiv  Detail & Related papers  (2024-12-10T15:50:43Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Conditional Korhunen-Lo\'{e}ve regression model with Basis Adaptation for high-dimensional problems: uncertainty quantification and inverse modeling [62.997667081978825]
  We propose a methodology for improving the accuracy of surrogate models of the observable response of physical systems. We apply the proposed methodology to constructing surrogate models via the Basis Adaptation (BA) method of the stationary hydraulic head response.
  arXiv  Detail & Related papers  (2023-07-05T18:14:38Z)
• Inverting brain grey matter models with likelihood-free inference: a tool for trustable cytoarchitecture measurements [62.997667081978825]
  characterisation of the brain grey matter cytoarchitecture with quantitative sensitivity to soma density and volume remains an unsolved challenge in dMRI. We propose a new forward model, specifically a new system of equations, requiring a few relatively sparse b-shells. We then apply modern tools from Bayesian analysis known as likelihood-free inference (LFI) to invert our proposed model.
  arXiv  Detail & Related papers  (2021-11-15T09:08:27Z)
• Scalable mixed-domain Gaussian process modeling and model reduction for longitudinal data [5.00301731167245]
  We derive a basis function approximation scheme for mixed-domain covariance functions.<n>We show that we can approximate the exact GP model accurately in a fraction of the runtime.<n>We also demonstrate a scalable model reduction workflow for obtaining smaller and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-03T04:47:37Z)
• A Data-driven feature selection and machine-learning model benchmark for the prediction of longitudinal dispersion coefficient [29.58577229101903]
  An accurate prediction on Longitudinal Dispersion(LD) coefficient can produce a performance leap in related simulation. In this study, a global optimal feature set was proposed through numerical comparison of the distilled local optimums in performance with representative ML models. Results show that the support vector machine has significantly better performance than other models.
  arXiv  Detail & Related papers  (2021-07-16T09:50:38Z)
• Understanding Implicit Regularization in Over-Parameterized Single Index Model [55.41685740015095]
  We design regularization-free algorithms for the high-dimensional single index model. We provide theoretical guarantees for the induced implicit regularization phenomenon.
  arXiv  Detail & Related papers  (2020-07-16T13:27:47Z)
• Non-parametric Models for Non-negative Functions [48.7576911714538]
  We provide the first model for non-negative functions from the same good linear models. We prove that it admits a representer theorem and provide an efficient dual formulation for convex problems.
  arXiv  Detail & Related papers  (2020-07-08T07:17:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.