Why Learning Rediscovers the Closed-Form Diagonal Regularizer

• URL: http://arxiv.org/abs/2609.09656v1
• Date: Wed, 09 Sep 2026 03:10:42 GMT
• Title: Why Learning Rediscovers the Closed-Form Diagonal Regularizer
• Abstract summary: When truncation noise is isotropic, Tikhonov shape is a closed-form power law Gamma_k proportional to where exponential_k|s| set by the prior alone.<n>Berry's random-wave conjecture decorrelates the truncation noise across modes, and Weyl's eigenvalue counting law supplies enough modes for the conclusion to survive empirical Berry violations.<n>On FEM-simulated acoustic rooms, the closed form is near-optimal relative to perroom tuning across observation windows, and three diagonal architectures trained on the same data match its oracle error within 1
• Score: 3.1822873305165618
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We identify a diagonal saturation principle in modal inverse problems: when truncation noise is isotropic, the Bayes-optimal Tikhonov shape is a closed-form power law Gamma_k proportional to lambda_k^|s| set by the prior alone, independent of the domain. Berry's random-wave conjecture decorrelates the truncation noise across modes, and Weyl's eigenvalue counting law supplies enough modes for the conclusion to survive empirical Berry violations. Together they predict an approximately flat loss landscape across the per-mode family, leaving narrow scope for a diagonal regularizer to robustly beat the closed form. On FEM-simulated acoustic rooms, the closed form is near-optimal relative to per-room oracle tuning across observation windows, and three diagonal architectures trained on the same data match its reconstruction error within 1 pp despite learning qualitatively different spectra. The framework extends to heat diffusion via a known exponential Green's function correction with no new free parameters. Saturation is restricted to the diagonal family: Learned Iterative Ridge crosses the boundary by exploiting cross-mode coupling, locating where learning starts to help.

Related papers

• Across the Loss Landscape with Progressive Growth [51.366966420881646]
  We show how grow-and-optimize strategies to train toward flatter regions by viewing growth as progressive relaxation.<n>We empirically validate these predictions in controlled toy basins and in a realistic ResNet/CI-100 setting.<n>Although progressive subspace growth reliably produces flatter solutions, curvature reductions do not universally translate into improved test performance, highlighting subtleties in the flatness-generalization connection.
  arXiv  Detail & Related papers  (2026-08-25T13:50:48Z)
• Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling [9.271408028342496]
  Constant-stepsize temporal-difference (TD) learning is attractive for policy evaluation, but inference from a single Markov trajectory must account for serial dependence and a stepsize-dependent stationary target.<n>We establish a functional central limit theorem whose covariance retains the multiplicative component induced by the random TD matrix and the stationary iterate error.<n>Experiments on FrozenLake and Garnet illustrate stationary-target coverage, RR target correction, and the finite-sample behavior of the horizon-indexed design.
  arXiv  Detail & Related papers  (2026-08-11T13:19:28Z)
• Certified boundary-magic witness for state-dependent proto-area in a holographic code [0.0]
  We study a four-qubit product-EPR holographic code whose reconstructing region contains matter and one leg of a geometry bond.<n>We show that only matter-controlled bond motion produces a leading quadratic logical-state dependence of the boundary entropy.
  arXiv  Detail & Related papers  (2026-07-12T09:16:46Z)
• Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data [3.7437735326653883]
  This paper studies the rate-controlled middle case for full-batch linear classification on separable data.<n>For memoryless stability-annealed smoothed-sign descent with weighted exponential loss, we prove that the normalized iterates converge to the minimizer of a convex Burg-type barrier over a margin slice.
  arXiv  Detail & Related papers  (2026-07-07T08:55:35Z)
• Orthogonality Edges in Strong-Coupling Quantum Work Statistics [0.5541461456285174]
  Strong inversion to a reservoir can do more than shift, broaden, or dress the work peaks of a driven quantum system.<n>We demonstrate this mechanism for the inclusive work distribution of the biased spin-boson model under a sudden bias inversion.
  arXiv  Detail & Related papers  (2026-07-04T16:53:05Z)
• Closed-Form Spectral Regularization for Multi-Task Model Merging [96.82449201305234]
  Model merging combines several independently fine-tuned experts into a single multi-task model without any training data.<n>State-of-the-art merging methods formulate merging as a layer-wise interference problem.<n>We propose SWUDI, a closed-form method that combines a soft exponential filter, which matches the gradient-flow trajectory of iterative descent.
  arXiv  Detail & Related papers  (2026-06-05T14:00:47Z)
• Dual-space posterior sampling for Bayesian inference in constrained inverse problems [5.739708455652395]
  Inverse problems constrained by partial differential equations are often ill-conditioned due to noisy and incomplete data or inherent non-uniqueness.<n>We develop a new technique for translating hard physical constraints, such as the wave equation, into prior distributions amenable to existing sampling techniques.
  arXiv  Detail & Related papers  (2026-02-28T00:34:31Z)
• Supervised Guidance Training for Infinite-Dimensional Diffusion Models [47.65586147952857]
  In inverse problems, the aim is to sample from a posterior distribution over functions obtained by conditioning a prior.<n>We prove that the models can be conditioned using an infinite-dimensional extension of Doob's $h$-transform.<n>We propose a simulation-free score matching objective (called Supervised Guidance Training) enabling efficient and stable posterior sampling.
  arXiv  Detail & Related papers  (2026-01-28T16:39:39Z)
• Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models [57.52124921268249]
  We propose a Trust Sequential Quadratic Programming method to find both first and second-order stationary points. To converge to first-order stationary points, our method computes a gradient step in each iteration defined by minimizing a approximation of the objective subject. To converge to second-order stationary points, our method additionally computes an eigen step to explore the negative curvature the reduced Hessian matrix.
  arXiv  Detail & Related papers  (2024-09-24T04:39:47Z)
• Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields [53.31927549039624]
  We show that a piecewise discretization preserves better contrast from existing discretization problems. We apply this theory to the problem of matching two images.
  arXiv  Detail & Related papers  (2021-07-13T12:31:06Z)
• Canny-VO: Visual Odometry with RGB-D Cameras based on Geometric 3D-2D Edge Alignment [85.32080531133799]
  This paper reviews the classical problem of free-form curve registration and applies it to an efficient RGBD visual odometry system called Canny-VO. Two replacements for the distance transformation commonly used in edge registration are proposed: Approximate Nearest Neighbour Fields and Oriented Nearest Neighbour Fields. 3D2D edge alignment benefits from these alternative formulations in terms of both efficiency and accuracy.
  arXiv  Detail & Related papers  (2020-12-15T11:42:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.