Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation

• URL: http://arxiv.org/abs/2605.01221v1
• Date: Sat, 02 May 2026 03:30:55 GMT
• Title: Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation
• Abstract summary: We propose Local Hessian Spectral Dimension (LHSD)<n>LHSD avoids full Hessian construction, achieving linear scalability with $D$.<n> Experiments on synthetic and real data confirm LHSD's superior robustness and its utility in detecting memorization in large-scale diffusion models.
• Score: 4.484170173286332
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While diffusion models enable new approaches for estimating Local Intrinsic Dimension (LID), existing methods fail in high-dimensional spaces where noise from vast normal directions overwhelms the tangent signal. We propose Local Hessian Spectral Dimension (LHSD), which resolves this by applying spectral filtering to the log-density Hessian, explicitly cutting off large eigenvalues associated with normal directions to count zero-curvature tangent directions. Implemented using Stochastic Lanczos Quadrature (SLQ), LHSD avoids full Hessian construction, achieving linear scalability with dimension $D$. Experiments on synthetic and real data confirm LHSD's superior robustness and its utility in detecting memorization in large-scale diffusion models.

Related papers

• Predicting When Random Low-Dimensional Reparameterizations Train Neural Networks [3.980892205755342]
  We present an orientation-resolved quadratic master formula that predicts the random-slice residual from both the curvature spectrum and the reference-to-solution displacement profile.<n>We then introduce Random Mapping Networks (RaMaN), which instantiate the predicted latent dimension using structured Hadamard or seed-regenerated Gaussian maps.<n>RaMaN closely tracks measured transition locations and outperforms orientation-agnostic approximations when displacement direction matters.
  arXiv  Detail & Related papers  (2026-08-12T21:09:54Z)
• Geometry Adaptive Counterfactual Distribution Learning with Diffusion-Guided Smoothing [4.727838288363022]
  Standard isotropic smoothing treats all ambient directions equally, leading to unfavorable scaling and unstable local inference.<n>We propose two diffusion-guided estimators based on semi-parametric debiasing: diffusion-informed smoothing for counterfactual densities and diffusion-informed score smoothing for counterfactual scores.
  arXiv  Detail & Related papers  (2026-05-25T13:02:56Z)
• Coverage Analysis of Rydberg Atom Quantum Receiver Arrays: A Stochastic Geometry Approach [55.47164192369501]
  Rydberg atomic quantum receivers (RAQRs) offer quantum-limited sensitivity and broadband tunability.<n>In dense deployments, aggregate interference can push atomic transducer out of its small-signal regime.<n>This paper addresses the question by embedding the RAQR front end into a geometry coverage analysis.
  arXiv  Detail & Related papers  (2026-05-22T04:05:13Z)
• Topological Residual Asymmetry for Bivariate Causal Direction [1.2691047660244335]
  Topological Residual Asymmetry is a geometry-based criterion for additive-noise models.<n>We quantify a bulk-tube contrast using a 0D persistent-homology functional.<n>Experiments across many challenging synthetic and real-data scenarios demonstrate the method's superiority.
  arXiv  Detail & Related papers  (2026-01-31T00:38:53Z)
• Evidence Slopes and Effective Dimension in Singular Linear Models [0.0]
  Singular learning theory replaces Laplace approximation with the real canonical threshold (RLCT)<n>We show theoretically and empirically that the error of Laplace/BIC times log n grows linearly with (d/2 minus) times log n.<n>Our results provide a concrete finite-sample characterization of Laplace failure in singular models and demonstrate that evidence slopes can be used as a practical estimator of effective dimension in simple linear settings.
  arXiv  Detail & Related papers  (2026-01-03T17:05:55Z)
• Robust Tangent Space Estimation via Laplacian Eigenvector Gradient Orthogonalization [48.25304391127552]
  Estimating the tangent spaces of a data manifold is a fundamental problem in data analysis.<n>We propose a method, Laplacian Eigenvector Gradient Orthogonalization (LEGO), that utilizes the global structure of the data to guide local tangent space estimation.
  arXiv  Detail & Related papers  (2025-10-02T17:59:45Z)
• Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality [21.10158431913811]
  We investigate how the DDPM can achieve sampling speed-ups through automatic exploitation of intrinsic low dimensionality of data. We prove that the iteration complexity of the DDPM scales nearly linearly with $k$, which is optimal when using KL divergence to measure distributional discrepancy.
  arXiv  Detail & Related papers  (2024-10-24T14:36:12Z)
• Closed-form Filtering for Non-linear Systems [83.91296397912218]
  We propose a new class of filters based on Gaussian PSD Models, which offer several advantages in terms of density approximation and computational efficiency. We show that filtering can be efficiently performed in closed form when transitions and observations are Gaussian PSD Models. Our proposed estimator enjoys strong theoretical guarantees, with estimation error that depends on the quality of the approximation and is adaptive to the regularity of the transition probabilities.
  arXiv  Detail & Related papers  (2024-02-15T08:51:49Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent [43.097493761380186]
  gradient algorithms are an efficient method of approximately solving linear systems. We show that gradient descent produces accurate predictions, even in cases where it does not converge quickly to the optimum. Experimentally, gradient descent achieves state-of-the-art performance on sufficiently large-scale or ill-conditioned regression tasks.
  arXiv  Detail & Related papers  (2023-06-20T15:07:37Z)
• Random Smoothing Regularization in Kernel Gradient Descent Learning [24.383121157277007]
  We present a framework for random smoothing regularization that can adaptively learn a wide range of ground truth functions belonging to the classical Sobolev spaces. Our estimator can adapt to the structural assumptions of the underlying data and avoid the curse of dimensionality.
  arXiv  Detail & Related papers  (2023-05-05T13:37:34Z)
• Intrinsic dimension estimation for discrete metrics [65.5438227932088]
  In this letter we introduce an algorithm to infer the intrinsic dimension (ID) of datasets embedded in discrete spaces. We demonstrate its accuracy on benchmark datasets, and we apply it to analyze a metagenomic dataset for species fingerprinting. This suggests that evolutive pressure acts on a low-dimensional manifold despite the high-dimensionality of sequences' space.
  arXiv  Detail & Related papers  (2022-07-20T06:38:36Z)
• Nonlinear Isometric Manifold Learning for Injective Normalizing Flows [58.720142291102135]
  We use isometries to separate manifold learning and density estimation. We also employ autoencoders to design embeddings with explicit inverses that do not distort the probability distribution.
  arXiv  Detail & Related papers  (2022-03-08T08:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.