On the Use of Bagging for Local Intrinsic Dimensionality Estimation

• URL: http://arxiv.org/abs/2603.24384v1
• Date: Wed, 25 Mar 2026 15:03:25 GMT
• Title: On the Use of Bagging for Local Intrinsic Dimensionality Estimation
• Abstract summary: We propose an ensemble approach that uses subbagging to preserve the local distribution of nearest neighbor (NN) distances.<n>We analyze both theoretically and experimentally how the choice of the sampling rate and the k-NN size used for LID estimation, alongside the ensemble size, affects performance.
• Score: 6.621518682026852
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The theory of Local Intrinsic Dimensionality (LID) has become a valuable tool for characterizing local complexity within and across data manifolds, supporting a range of data mining and machine learning tasks. Accurate LID estimation requires samples drawn from small neighborhoods around each query to avoid biases from nonlocal effects and potential manifold mixing, yet limited data within such neighborhoods tends to cause high estimation variance. As a variance reduction strategy, we propose an ensemble approach that uses subbagging to preserve the local distribution of nearest neighbor (NN) distances. The main challenge is that the uniform reduction in total sample size within each subsample increases the proximity threshold for finding a fixed number k of NNs around the query. As a result, in the specific context of LID estimation, the sampling rate has an additional, complex interplay with the neighborhood size, where both combined determine the sample size as well as the locality and resolution considered for estimation. We analyze both theoretically and experimentally how the choice of the sampling rate and the k-NN size used for LID estimation, alongside the ensemble size, affects performance, enabling informed prior selection of these hyper-parameters depending on application-based preferences. Our results indicate that within broad and well-characterized regions of the hyper-parameters space, using a bagged estimator will most often significantly reduce variance as well as the mean squared error when compared to the corresponding non-bagged baseline, with controllable impact on bias. We additionally propose and evaluate different ways of combining bagging with neighborhood smoothing for substantial further improvements on LID estimation performance.

Related papers

• Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport [88.55403215843967]
  Voronoi-based partial optimal transport (VPOT) for obtaining refined local and global convergence guarantees for the maximum likelihood estimator of the mixing measure.<n>Results hold in arbitrary fixed dimension without requiring mixing proportions to be uniformly bounded away from zero or prior knowledge of the true number of mixture components.
  arXiv  Detail & Related papers  (2026-09-15T04:32:53Z)
• Breaking the Curse with BAND: Nonparametric Distribution Estimation in High Dimensions [1.338174941551702]
  We propose a sparse Bayesian network approach in which each conditional probability is estimated using sparsity-aware conditional mean methods.<n>The resulting estimator, textitBAyesian Network Distribution (BAND) handles mixed data types in high-dimensional time series.<n>BAND performs competitively for data sampling and confidence region forecasting against a range of state-of-the-art benchmarks.
  arXiv  Detail & Related papers  (2026-07-29T14:21:39Z)
• Efficient Covariance Estimation for Sparsified Functional Data [51.69796254617083]
  proposed Random-knots (Random-knots-Spatial) and B-spline (Bspline-Spatial) estimators of the covariance function are computationally efficient.<n>Asymptotic pointwise of the covariance are obtained for sparsified individual trajectories under some regularity conditions.
  arXiv  Detail & Related papers  (2025-11-23T00:50:33Z)
• Towards Size-invariant Salient Object Detection: A Generic Evaluation and Optimization Approach [118.75896764188424]
  We present a novel perspective to expose the inherent size sensitivity of existing widely used Salient Object Detection metrics.<n>To address this challenge, a generic Size-Invariant Evaluation (SIEva) framework is proposed.<n>We further develop a dedicated optimization framework (SIOpt), which adheres to the size-invariant principle and significantly enhances the detection of salient objects across a broad range of sizes.
  arXiv  Detail & Related papers  (2025-09-19T04:12:14Z)
• Bayesian Multivariate Density-Density Regression [25.35298354797079]
  We introduce a novel and scalable Bayesian framework for multivariate-density-density regression (DDR)<n>Our approach addresses the critical issue of distributions residing in spaces of differing dimensions.<n>We show that Bayesian DDR provides robust fits, superior predictive performance compared to traditional methods, and valuable insights into complex biological interactions.
  arXiv  Detail & Related papers  (2025-04-17T03:46:03Z)
• Stability and Generalization for Distributed SGDA [70.97400503482353]
  We propose the stability-based generalization analytical framework for Distributed-SGDA. We conduct a comprehensive analysis of stability error, generalization gap, and population risk across different metrics. Our theoretical results reveal the trade-off between the generalization gap and optimization error.
  arXiv  Detail & Related papers  (2024-11-14T11:16:32Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• An analysis of the noise schedule for score-based generative models [7.180235086275926]
  Score-based generative models (SGMs) aim at estimating a target data distribution by learning score functions using only noise-perturbed samples from the target.<n>Recent literature has focused extensively on assessing the error between the target and estimated distributions, gauging the generative quality through the Kullback-Leibler (KL) divergence and Wasserstein distances.<n>We establish an upper bound for the KL divergence between the target and the estimated distributions, explicitly depending on any time-dependent noise schedule.
  arXiv  Detail & Related papers  (2024-02-07T08:24:35Z)
• Far Away in the Deep Space: Dense Nearest-Neighbor-Based Out-of-Distribution Detection [33.78080060234557]
  Nearest-Neighbors approaches have been shown to work well in object-centric data domains. We show that nearest-neighbor approaches also yield state-of-the-art results on dense novelty detection in complex driving scenes.
  arXiv  Detail & Related papers  (2022-11-12T13:32:19Z)
• Intrinsic Dimensionality Estimation within Tight Localities: A Theoretical and Experimental Analysis [0.0]
  We propose a local ID estimation strategy stable even for tight' localities consisting of as few as 20 sample points. Our experimental results show that our proposed estimation technique can achieve notably smaller variance, while maintaining comparable levels of bias, at much smaller sample sizes than state-of-the-art estimators.
  arXiv  Detail & Related papers  (2022-09-29T00:00:11Z)
• Meta Learning Low Rank Covariance Factors for Energy-Based Deterministic Uncertainty [58.144520501201995]
  Bi-Lipschitz regularization of neural network layers preserve relative distances between data instances in the feature spaces of each layer. With the use of an attentive set encoder, we propose to meta learn either diagonal or diagonal plus low-rank factors to efficiently construct task specific covariance matrices. We also propose an inference procedure which utilizes scaled energy to achieve a final predictive distribution.
  arXiv  Detail & Related papers  (2021-10-12T22:04:19Z)
• Distributionally Robust Local Non-parametric Conditional Estimation [22.423052432220235]
  We propose a new distributionally robust estimator that generates non-parametric local estimates. We show that despite being generally intractable, the local estimator can be efficiently found via convex optimization. Experiments with synthetic and MNIST datasets show the competitive performance of this new class of estimators.
  arXiv  Detail & Related papers  (2020-10-12T00:11:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.