Stochastic and Private Nonconvex Outlier-Robust PCA

• URL: http://arxiv.org/abs/2203.09276v1
• Date: Thu, 17 Mar 2022 12:00:47 GMT
• Title: Stochastic and Private Nonconvex Outlier-Robust PCA
• Authors: Tyler Maunu, Chenyu Yu, Gilad Lerman
• Abstract summary: Outlier-robust PCA seeks an underlying low-dimensional linear subspace from a dataset corrupted with outliers. We show that our methods involve our methods, which involve a geodesic descent and a novel convergence analysis. The main application method is an effectively private algorithm for outlier-robust PCA.
• Score: 11.688030627514532
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop theoretically guaranteed stochastic methods for outlier-robust PCA. Outlier-robust PCA seeks an underlying low-dimensional linear subspace from a dataset that is corrupted with outliers. We are able to show that our methods, which involve stochastic geodesic gradient descent over the Grassmannian manifold, converge and recover an underlying subspace in various regimes through the development of a novel convergence analysis. The main application of this method is an effective differentially private algorithm for outlier-robust PCA that uses a Gaussian noise mechanism within the stochastic gradient method. Our results emphasize the advantages of the nonconvex methods over another convex approach to solving this problem in the differentially private setting. Experiments on synthetic and stylized data verify these results.

Related papers

• Assumption-Lean Post-Integrated Inference with Negative Control Outcomes [0.0]
  We introduce a robust post-integrated inference (PII) method that adjusts for latent heterogeneity using negative control outcomes. Our method extends to projected direct effect estimands, accounting for hidden mediators, confounders, and moderators. The proposed doubly robust estimators are consistent and efficient under minimal assumptions and potential misspecification.
  arXiv  Detail & Related papers  (2024-10-07T12:52:38Z)
• Total Uncertainty Quantification in Inverse PDE Solutions Obtained with Reduced-Order Deep Learning Surrogate Models [50.90868087591973]
  We propose an approximate Bayesian method for quantifying the total uncertainty in inverse PDE solutions obtained with machine learning surrogate models. We test the proposed framework by comparing it with the iterative ensemble smoother and deep ensembling methods for a non-linear diffusion equation.
  arXiv  Detail & Related papers  (2024-08-20T19:06:02Z)
• Study of Robust Direction Finding Based on Joint Sparse Representation [2.3333781137726137]
  We propose a novel DOA estimation method based on sparse signal recovery (SSR) To address the issue of grid mismatch, we utilize an alternating optimization approach. Simulation results demonstrate that the proposed method exhibits robustness against large outliers.
  arXiv  Detail & Related papers  (2024-05-27T02:26:37Z)
• A Unified Theory of Stochastic Proximal Point Methods without Smoothness [52.30944052987393]
  Proximal point methods have attracted considerable interest owing to their numerical stability and robustness against imperfect tuning. This paper presents a comprehensive analysis of a broad range of variations of the proximal point method (SPPM)
  arXiv  Detail & Related papers  (2024-05-24T21:09:19Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• Robust Stochastic Optimization via Gradient Quantile Clipping [6.2844649973308835]
  We introduce a quant clipping strategy for Gradient Descent (SGD) We use gradient new outliers as norm clipping chains. We propose an implementation of the algorithm using Huberiles.
  arXiv  Detail & Related papers  (2023-09-29T15:24:48Z)
• ALPCAH: Sample-wise Heteroscedastic PCA with Tail Singular Value Regularization [17.771454131646312]
  Principal component analysis is a key tool in the field of data dimensionality reduction. This paper develops a PCA method that can estimate the sample-wise noise variances. It is done without distributional assumptions of the low-rank component and without assuming the noise variances are known.
  arXiv  Detail & Related papers  (2023-07-06T03:11:11Z)
• CoMadOut -- A Robust Outlier Detection Algorithm based on CoMAD [0.3749861135832073]
  Outliers play a significant role, since they bear the potential to distort the predictions of a machine learning algorithm on a given dataset. To address this problem, we propose the robust outlier detection algorithm CoMadOut. Our approach can be seen as a robust alternative for outlier detection tasks.
  arXiv  Detail & Related papers  (2022-11-23T21:33:34Z)
• Posterior and Computational Uncertainty in Gaussian Processes [52.26904059556759]
  Gaussian processes scale prohibitively with the size of the dataset. Many approximation methods have been developed, which inevitably introduce approximation error. This additional source of uncertainty, due to limited computation, is entirely ignored when using the approximate posterior. We develop a new class of methods that provides consistent estimation of the combined uncertainty arising from both the finite number of data observed and the finite amount of computation expended.
  arXiv  Detail & Related papers  (2022-05-30T22:16:25Z)
• Differentiable Annealed Importance Sampling and the Perils of Gradient Noise [68.44523807580438]
  Annealed importance sampling (AIS) and related algorithms are highly effective tools for marginal likelihood estimation. Differentiability is a desirable property as it would admit the possibility of optimizing marginal likelihood as an objective. We propose a differentiable algorithm by abandoning Metropolis-Hastings steps, which further unlocks mini-batch computation.
  arXiv  Detail & Related papers  (2021-07-21T17:10:14Z)
• Scalable Variational Gaussian Processes via Harmonic Kernel Decomposition [54.07797071198249]
  We introduce a new scalable variational Gaussian process approximation which provides a high fidelity approximation while retaining general applicability. We demonstrate that, on a range of regression and classification problems, our approach can exploit input space symmetries such as translations and reflections. Notably, our approach achieves state-of-the-art results on CIFAR-10 among pure GP models.
  arXiv  Detail & Related papers  (2021-06-10T18:17:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.