Free Decompression with Algebraic Spectral Curves

• URL: http://arxiv.org/abs/2605.03634v1
• Date: Tue, 05 May 2026 11:03:48 GMT
• Title: Free Decompression with Algebraic Spectral Curves
• Abstract summary: Free decompression (FD) is a recently proposed method for extrapolating spectral information across matrix sizes.<n>This paper recasts FD as an evolution along spectral curves which can be readily integrated.<n>We demonstrate the efficacy of our framework on models of interest in modern machine learning.
• Score: 45.17701640697655
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tools from random matrix theory have become central to deep learning theory, using spectral information to provide mechanisms for modeling generalization, robustness, scaling, and failure modes. While often capable of modeling empirical behavior, practical computations are limited by matrix size, often imposing a restriction to models that are too small to be realistic. This motivates the inference of properties of larger models from the behavior of smaller ones. Free decompression (FD) is a recently proposed method for extrapolating spectral information across matrix sizes, but its utility is currently limited by strong assumptions that preclude its implementation on more realistic machine learning (ML) models. We use algebraic spectral curve theory to provide a general FD methodology for spectral densities whose Stieltjes transform satisfies an algebraic relation, a modeling assumption that is more likely to hold in practice. This recasts FD as an evolution along spectral curves which can be readily integrated. Our framework enables the expansion of spectral densities that have multiple or multi-modal bulks, that exist at multiple scales, and that contain atoms, all characteristic of real-world data and popular ML models. We demonstrate the efficacy of our framework on models of interest in modern ML, including Hessian and activation matrices associated with neural networks and large-scale diffusion models.

Related papers

• Data-Driven Spectral Prediction for Accelerating Large-Scale Electronic Structure Calculations [0.0]
  We investigate three machine learning models trained on a novel 2 TB dataset of protein dimers.<n>The predicted spectra provide initial guesses that effectively bypass early Self-Consistent Field iterations in BigDFT.<n>These spectral predictors will be deployed to dynamically optimize upcoming rational filter-based eigensolvers, such as FrASE, which is currently in initial development.
  arXiv  Detail & Related papers  (2026-05-29T22:35:15Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Random Matrix Theory for Deep Learning: Beyond Eigenvalues of Linear Models [51.85815025140659]
  Modern Machine Learning (ML) and Deep Neural Networks (DNNs) often operate on high-dimensional data.<n>In particular, the proportional regime where the data dimension, sample size, and number of model parameters are all large gives rise to novel and sometimes counterintuitive behaviors.<n>This paper extends traditional Random Matrix Theory (RMT) beyond eigenvalue-based analysis of linear models to address the challenges posed by nonlinear ML models.
  arXiv  Detail & Related papers  (2025-06-16T06:54:08Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Preventing Model Collapse in Gaussian Process Latent Variable Models [11.45681373843122]
  This paper theoretically examines the impact of projection variance on model collapse through the lens of a linear FourierVM. We tackle model collapse due to inadequate kernel flexibility by integrating the spectral mixture (SM) kernel and a differentiable random feature (RFF) kernel approximation. The proposedVM, named advisedRFLVM, is evaluated across diverse datasets and consistently outperforms various competing models.
  arXiv  Detail & Related papers  (2024-04-02T06:58:41Z)
• Invertible Low-Dimensional Modelling of X-ray Absorption Spectra for Potential Applications in Spectral X-ray Imaging [0.0]
  We propose a novel, non-linear model that combines a deep neural network autoencoder with an optimal linear model based on the Singular Value Decomposition (SVD) We demonstrate the advantages of our method over traditional models, especially when modelling X-ray absorption spectra that contain K-edges in the energy range of interest.
  arXiv  Detail & Related papers  (2023-07-10T11:14:19Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• Information-theoretic limits of a multiview low-rank symmetric spiked matrix model [19.738567726658875]
  We consider a generalization of an important class of high-dimensional inference problems, namely spiked symmetric matrix models. We rigorously establish the information-theoretic limits through the proof of single-letter formulas. We improve the recently introduced adaptive method, so that it can be used to study low-rank models.
  arXiv  Detail & Related papers  (2020-05-16T15:31:07Z)
• Explainable Matrix -- Visualization for Global and Local Interpretability of Random Forest Classification Ensembles [78.6363825307044]
  We propose Explainable Matrix (ExMatrix), a novel visualization method for Random Forest (RF) interpretability. It employs a simple yet powerful matrix-like visual metaphor, where rows are rules, columns are features, and cells are rules predicates. ExMatrix applicability is confirmed via different examples, showing how it can be used in practice to promote RF models interpretability.
  arXiv  Detail & Related papers  (2020-05-08T21:03:48Z)
• Dimension Independent Generalization Error by Stochastic Gradient Descent [12.474236773219067]
  We present a theory on the generalization error of descent (SGD) solutions for both and locally convex loss functions. We show that the generalization error does not depend on the $p$ dimension or depends on the low effective $p$logarithmic factor.
  arXiv  Detail & Related papers  (2020-03-25T03:08:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.