Self-paced Principal Component Analysis
- URL: http://arxiv.org/abs/2106.13880v1
- Date: Fri, 25 Jun 2021 20:50:45 GMT
- Title: Self-paced Principal Component Analysis
- Authors: Zhao Kang, Hongfei Liu, Jiangxin Li, Xiaofeng Zhu, and Ling Tian
- Abstract summary: We propose a novel method called Self-paced PCA (SPCA) to further reduce the effect of noise and outliers.
The complexity of each sample is calculated at the beginning of each iteration in order to integrate samples from simple to more complex into training.
- Score: 17.333976289539457
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Principal Component Analysis (PCA) has been widely used for dimensionality
reduction and feature extraction. Robust PCA (RPCA), under different robust
distance metrics, such as l1-norm and l2, p-norm, can deal with noise or
outliers to some extent. However, real-world data may display structures that
can not be fully captured by these simple functions. In addition, existing
methods treat complex and simple samples equally. By contrast, a learning
pattern typically adopted by human beings is to learn from simple to complex
and less to more. Based on this principle, we propose a novel method called
Self-paced PCA (SPCA) to further reduce the effect of noise and outliers.
Notably, the complexity of each sample is calculated at the beginning of each
iteration in order to integrate samples from simple to more complex into
training. Based on an alternating optimization, SPCA finds an optimal
projection matrix and filters out outliers iteratively. Theoretical analysis is
presented to show the rationality of SPCA. Extensive experiments on popular
data sets demonstrate that the proposed method can improve the state of-the-art
results considerably.
Related papers
- Efficient Estimation of Unique Components in Independent Component Analysis by Matrix Representation [1.0282274843007793]
Independent component analysis (ICA) is a widely used method in various applications of signal processing and feature extraction.
In this paper, the unique estimation of ICA is highly accelerated by reformulating the algorithm in matrix representation.
Experimental results on artificial datasets and EEG data verified the efficiency of the proposed method.
arXiv Detail & Related papers (2024-08-30T09:01:04Z) - Fast Shapley Value Estimation: A Unified Approach [71.92014859992263]
We propose a straightforward and efficient Shapley estimator, SimSHAP, by eliminating redundant techniques.
In our analysis of existing approaches, we observe that estimators can be unified as a linear transformation of randomly summed values from feature subsets.
Our experiments validate the effectiveness of our SimSHAP, which significantly accelerates the computation of accurate Shapley values.
arXiv Detail & Related papers (2023-11-02T06:09:24Z) - Entropic Wasserstein Component Analysis [8.744017403796406]
A key requirement for Dimension reduction (DR) is to incorporate global dependencies among original and embedded samples.
We combine the principles of optimal transport (OT) and principal component analysis (PCA)
Our method seeks the best linear subspace that minimizes reconstruction error using entropic OT, which naturally encodes the neighborhood information of the samples.
arXiv Detail & Related papers (2023-03-09T08:59:33Z) - Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data.
We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
arXiv Detail & Related papers (2023-03-06T14:49:59Z) - Asymptotically Unbiased Instance-wise Regularized Partial AUC
Optimization: Theory and Algorithm [101.44676036551537]
One-way Partial AUC (OPAUC) and Two-way Partial AUC (TPAUC) measures the average performance of a binary classifier.
Most of the existing methods could only optimize PAUC approximately, leading to inevitable biases that are not controllable.
We present a simpler reformulation of the PAUC problem via distributional robust optimization AUC.
arXiv Detail & Related papers (2022-10-08T08:26:22Z) - Enhanced Principal Component Analysis under A Collaborative-Robust
Framework [89.28334359066258]
We introduce a general collaborative-robust weight learning framework that combines weight learning and robust loss in a non-trivial way.
Under the proposed framework, only a part of well-fitting samples are activated which indicates more importance during training, and others, whose errors are large, will not be ignored.
In particular, the negative effects of inactivated samples are alleviated by the robust loss function.
arXiv Detail & Related papers (2021-03-22T15:17:37Z) - Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature
Selection [138.97647716793333]
We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization.
We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
arXiv Detail & Related papers (2020-12-29T04:08:38Z) - Investigating the Scalability and Biological Plausibility of the
Activation Relaxation Algorithm [62.997667081978825]
Activation Relaxation (AR) algorithm provides a simple and robust approach for approximating the backpropagation of error algorithm.
We show that the algorithm can be further simplified and made more biologically plausible by introducing a learnable set of backwards weights.
We also investigate whether another biologically implausible assumption of the original AR algorithm -- the frozen feedforward pass -- can be relaxed without damaging performance.
arXiv Detail & Related papers (2020-10-13T08:02:38Z) - Principal Ellipsoid Analysis (PEA): Efficient non-linear dimension
reduction & clustering [9.042239247913642]
This article focuses on improving upon PCA and k-means, by allowing nonlinear relations in the data and more flexible cluster shapes.
The key contribution is a new framework for Principal Analysis (PEA), defining a simple and computationally efficient alternative to PCA.
In a rich variety of real data clustering applications, PEA is shown to do as well as k-means for simple datasets, while dramatically improving performance in more complex settings.
arXiv Detail & Related papers (2020-08-17T06:25:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.