Nearly-Linear Time and Streaming Algorithms for Outlier-Robust PCA

• URL: http://arxiv.org/abs/2305.02544v1
• Date: Thu, 4 May 2023 04:45:16 GMT
• Title: Nearly-Linear Time and Streaming Algorithms for Outlier-Robust PCA
• Authors: Ilias Diakonikolas, Daniel M. Kane, Ankit Pensia, Thanasis Pittas
• Abstract summary: We develop a nearly-linear time algorithm for robust PCA with near-optimal error guarantees. We also develop a single-pass streaming algorithm for robust PCA with memory usage nearly-linear in the dimension.
• Score: 43.106438224356175
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study principal component analysis (PCA), where given a dataset in $\mathbb{R}^d$ from a distribution, the task is to find a unit vector $v$ that approximately maximizes the variance of the distribution after being projected along $v$. Despite being a classical task, standard estimators fail drastically if the data contains even a small fraction of outliers, motivating the problem of robust PCA. Recent work has developed computationally-efficient algorithms for robust PCA that either take super-linear time or have sub-optimal error guarantees. Our main contribution is to develop a nearly-linear time algorithm for robust PCA with near-optimal error guarantees. We also develop a single-pass streaming algorithm for robust PCA with memory usage nearly-linear in the dimension.

Related papers

• Improved Privacy-Preserving PCA Using Optimized Homomorphic Matrix Multiplication [0.0]
  Principal Component Analysis (PCA) is a pivotal technique widely utilized in the realms of machine learning and data analysis. In recent years, there have been endeavors to utilize homomorphic encryption in privacy-preserving PCA algorithms for the secure cloud computing scenario. We propose a novel approach to privacy-preserving PCA that addresses these limitations, resulting in superior efficiency, accuracy, and scalability compared to previous approaches.
  arXiv  Detail & Related papers  (2023-05-27T02:51:20Z)
• Streaming Kernel PCA Algorithm With Small Space [24.003544967343615]
  Streaming PCA has gained significant attention in recent years, as it can handle large datasets efficiently. We propose a streaming algorithm for Kernel problems based on the traditional scheme by Oja. Our algorithm addresses the challenge of reducing the memory usage of PCA while maintaining its accuracy.
  arXiv  Detail & Related papers  (2023-03-08T13:13:33Z)
• Sparse PCA on fixed-rank matrices [0.05076419064097732]
  We show that, if the rank of the covariance matrix is a fixed value, then there is an algorithm that solves sparse PCA to global optimality. We also prove a similar result for the version of sparse PCA which requires the principal components to have disjoint supports.
  arXiv  Detail & Related papers  (2022-01-07T15:05:32Z)
• Robust factored principal component analysis for matrix-valued outlier accommodation and detection [4.228971753938522]
  Factored PCA (FPCA) is a probabilistic extension of PCA for matrix data. We propose a robust extension of FPCA (RFPCA) for matrix data. RFPCA can adaptively down-weight outliers and yield robust estimates.
  arXiv  Detail & Related papers  (2021-12-13T16:12:22Z)
• AgFlow: Fast Model Selection of Penalized PCA via Implicit Regularization Effects of Gradient Flow [64.81110234990888]
  Principal component analysis (PCA) has been widely used as an effective technique for feature extraction and dimension reduction. In the High Dimension Low Sample Size (HDLSS) setting, one may prefer modified principal components, with penalized loadings. We propose Approximated Gradient Flow (AgFlow) as a fast model selection method for penalized PCA.
  arXiv  Detail & Related papers  (2021-10-07T08:57:46Z)
• Outlier-Robust Sparse Estimation via Non-Convex Optimization [73.18654719887205]
  We explore the connection between high-dimensional statistics and non-robust optimization in the presence of sparsity constraints. We develop novel and simple optimization formulations for these problems. As a corollary, we obtain that any first-order method that efficiently converges to station yields an efficient algorithm for these tasks.
  arXiv  Detail & Related papers  (2021-09-23T17:38:24Z)
• FAST-PCA: A Fast and Exact Algorithm for Distributed Principal Component Analysis [12.91948651812873]
  Principal Component Analysis (PCA) is a fundamental data preprocessing tool in the world of machine learning. This paper proposes a distributed PCA algorithm called FAST-PCA (Fast and exAct diSTributed PCA)
  arXiv  Detail & Related papers  (2021-08-27T16:10:59Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)
• List-Decodable Mean Estimation in Nearly-PCA Time [50.79691056481693]
  We study the fundamental task of list-decodable mean estimation in high dimensions. Our algorithm runs in time $widetildeO(ndk)$ for all $k = O(sqrtd) cup Omega(d)$, where $n$ is the size of the dataset. A variant of our algorithm has runtime $widetildeO(ndk)$ for all $k$, at the expense of an $O(sqrtlog k)$ factor in the recovery guarantee
  arXiv  Detail & Related papers  (2020-11-19T17:21:37Z)
• Approximation Algorithms for Sparse Principal Component Analysis [57.5357874512594]
  Principal component analysis (PCA) is a widely used dimension reduction technique in machine learning and statistics. Various approaches to obtain sparse principal direction loadings have been proposed, which are termed Sparse Principal Component Analysis. We present thresholding as a provably accurate, time, approximation algorithm for the SPCA problem.
  arXiv  Detail & Related papers  (2020-06-23T04:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.