Estimating Shape Distances on Neural Representations with Limited Samples

• URL: http://arxiv.org/abs/2310.05742v2
• Date: Sat, 9 Dec 2023 21:44:23 GMT
• Title: Estimating Shape Distances on Neural Representations with Limited Samples
• Authors: Dean A. Pospisil, Brett W. Larsen, Sarah E. Harvey, Alex H. Williams
• Abstract summary: We develop a rigorous statistical theory for high-dimensional shape estimation. We show that this estimator achieves lower bias than on neurals simulation data, particularly in high-dimensional settings.
• Score: 5.959815242044236
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Measuring geometric similarity between high-dimensional network representations is a topic of longstanding interest to neuroscience and deep learning. Although many methods have been proposed, only a few works have rigorously analyzed their statistical efficiency or quantified estimator uncertainty in data-limited regimes. Here, we derive upper and lower bounds on the worst-case convergence of standard estimators of shape distance$\unicode{x2014}$a measure of representational dissimilarity proposed by Williams et al. (2021).These bounds reveal the challenging nature of the problem in high-dimensional feature spaces. To overcome these challenges, we introduce a new method-of-moments estimator with a tunable bias-variance tradeoff. We show that this estimator achieves substantially lower bias than standard estimators in simulation and on neural data, particularly in high-dimensional settings. Thus, we lay the foundation for a rigorous statistical theory for high-dimensional shape analysis, and we contribute a new estimation method that is well-suited to practical scientific settings.

Related papers

• Intrinsic Dimensionality Estimation within Tight Localities: A Theoretical and Experimental Analysis [0.0]
  We propose a local ID estimation strategy stable even for tight' localities consisting of as few as 20 sample points. Our experimental results show that our proposed estimation technique can achieve notably smaller variance, while maintaining comparable levels of bias, at much smaller sample sizes than state-of-the-art estimators.
  arXiv  Detail & Related papers  (2022-09-29T00:00:11Z)
• DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models [11.917949887615567]
  We propose DeepBayes estimators that leverage the power of deep recurrent neural networks in learning an estimator. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-05-04T18:12:17Z)
• On the Minimal Adversarial Perturbation for Deep Neural Networks with Provable Estimation Error [65.51757376525798]
  The existence of adversarial perturbations has opened an interesting research line on provable robustness. No provable results have been presented to estimate and bound the error committed. This paper proposes two lightweight strategies to find the minimal adversarial perturbation. The obtained results show that the proposed strategies approximate the theoretical distance and robustness for samples close to the classification, leading to provable guarantees against any adversarial attacks.
  arXiv  Detail & Related papers  (2022-01-04T16:40:03Z)
• Near-optimal inference in adaptive linear regression [60.08422051718195]
  Even simple methods like least squares can exhibit non-normal behavior when data is collected in an adaptive manner. We propose a family of online debiasing estimators to correct these distributional anomalies in at least squares estimation. We demonstrate the usefulness of our theory via applications to multi-armed bandit, autoregressive time series estimation, and active learning with exploration.
  arXiv  Detail & Related papers  (2021-07-05T21:05:11Z)
• Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral [58.434753643798224]
  Divergence frontiers have been proposed as an evaluation framework for generative models. We establish non-asymptotic bounds on the sample complexity of the plug-in estimator of divergence frontiers. We also augment the divergence frontier framework by investigating the statistical performance of smoothed distribution estimators.
  arXiv  Detail & Related papers  (2021-06-15T06:26:25Z)
• Intrinsic Dimension Estimation [92.87600241234344]
  We introduce a new estimator of the intrinsic dimension and provide finite sample, non-asymptotic guarantees. We then apply our techniques to get new sample complexity bounds for Generative Adversarial Networks (GANs) depending on the intrinsic dimension of the data.
  arXiv  Detail & Related papers  (2021-06-08T00:05:39Z)
• Minimax Estimation of Conditional Moment Models [40.95498063465325]
  We introduce a min-max criterion function, under which the estimation problem can be thought of as solving a zero-sum game. We analyze the statistical estimation rate of the resulting estimator for arbitrary hypothesis spaces. We show how our modified mean squared error rate, combined with conditions that bound the ill-posedness of the inverse problem, lead to mean squared error rates.
  arXiv  Detail & Related papers  (2020-06-12T14:02:38Z)
• Deep Dimension Reduction for Supervised Representation Learning [51.10448064423656]
  We propose a deep dimension reduction approach to learning representations with essential characteristics. The proposed approach is a nonparametric generalization of the sufficient dimension reduction method. We show that the estimated deep nonparametric representation is consistent in the sense that its excess risk converges to zero.
  arXiv  Detail & Related papers  (2020-06-10T14:47:43Z)
• Instability, Computational Efficiency and Statistical Accuracy [101.32305022521024]
  We develop a framework that yields statistical accuracy based on interplay between the deterministic convergence rate of the algorithm at the population level, and its degree of (instability) when applied to an empirical object based on $n$ samples. We provide applications of our general results to several concrete classes of models, including Gaussian mixture estimation, non-linear regression models, and informative non-response models.
  arXiv  Detail & Related papers  (2020-05-22T22:30:52Z)
• An Optimal Statistical and Computational Framework for Generalized Tensor Estimation [10.899518267165666]
  This paper describes a flexible framework for low-rank tensor estimation problems. It includes many important instances from applications in computational imaging, genomics, and network analysis.
  arXiv  Detail & Related papers  (2020-02-26T01:54:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.