Sparse PCA with False Discovery Rate Controlled Variable Selection
- URL: http://arxiv.org/abs/2401.08375v1
- Date: Tue, 16 Jan 2024 14:07:36 GMT
- Title: Sparse PCA with False Discovery Rate Controlled Variable Selection
- Authors: Jasin Machkour, Arnaud Breloy, Michael Muma, Daniel P. Palomar,
Fr\'ed\'eric Pascal
- Abstract summary: We propose an alternative formulation of sparse PCA driven by the false discovery rate (FDR)
A major advantage of the resulting T-Rex PCA is that no sparsity parameter tuning is required.
Numerical experiments and a stock market data example demonstrate a significant performance improvement.
- Score: 12.167049432063129
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Sparse principal component analysis (PCA) aims at mapping large dimensional
data to a linear subspace of lower dimension. By imposing loading vectors to be
sparse, it performs the double duty of dimension reduction and variable
selection. Sparse PCA algorithms are usually expressed as a trade-off between
explained variance and sparsity of the loading vectors (i.e., number of
selected variables). As a high explained variance is not necessarily synonymous
with relevant information, these methods are prone to select irrelevant
variables. To overcome this issue, we propose an alternative formulation of
sparse PCA driven by the false discovery rate (FDR). We then leverage the
Terminating-Random Experiments (T-Rex) selector to automatically determine an
FDR-controlled support of the loading vectors. A major advantage of the
resulting T-Rex PCA is that no sparsity parameter tuning is required. Numerical
experiments and a stock market data example demonstrate a significant
performance improvement.
Related papers
- DPVIm: Differentially Private Variational Inference Improved [13.761202518891329]
Differentially private (DP) release of multidimensional statistics typically considers an aggregate sensitivity.
Different dimensions of that vector might have widely different magnitudes and therefore DP perturbation disproportionately affects the signal across dimensions.
We observe this problem in the gradient release of the DP-SGD algorithm when using it for variational inference (VI)
arXiv Detail & Related papers (2022-10-28T07:41:32Z) - Robust factored principal component analysis for matrix-valued outlier
accommodation and detection [4.228971753938522]
Factored PCA (FPCA) is a probabilistic extension of PCA for matrix data.
We propose a robust extension of FPCA (RFPCA) for matrix data.
RFPCA can adaptively down-weight outliers and yield robust estimates.
arXiv Detail & Related papers (2021-12-13T16:12:22Z) - Determinantal point processes based on orthogonal polynomials for
sampling minibatches in SGD [0.0]
gradient descent (SGD) is a cornerstone of machine learning.
default minibatch construction involves uniformly sampling a subset of the desired size.
We show how specific DPPs and a string of controlled approximations can lead to gradient estimators with a variance that decays faster with the batchsize than under uniform sampling.
arXiv Detail & Related papers (2021-12-11T15:09:19Z) - The Terminating-Random Experiments Selector: Fast High-Dimensional
Variable Selection with False Discovery Rate Control [10.86851797584794]
T-Rex selector controls a user-defined target false discovery rate (FDR)
Experiments are conducted on a combination of the original predictors and multiple sets of randomly generated dummy predictors.
arXiv Detail & Related papers (2021-10-12T14:52:46Z) - AgFlow: Fast Model Selection of Penalized PCA via Implicit
Regularization Effects of Gradient Flow [64.81110234990888]
Principal component analysis (PCA) has been widely used as an effective technique for feature extraction and dimension reduction.
In the High Dimension Low Sample Size (HDLSS) setting, one may prefer modified principal components, with penalized loadings.
We propose Approximated Gradient Flow (AgFlow) as a fast model selection method for penalized PCA.
arXiv Detail & Related papers (2021-10-07T08:57:46Z) - Weight Vector Tuning and Asymptotic Analysis of Binary Linear
Classifiers [82.5915112474988]
This paper proposes weight vector tuning of a generic binary linear classifier through the parameterization of a decomposition of the discriminant by a scalar.
It is also found that weight vector tuning significantly improves the performance of Linear Discriminant Analysis (LDA) under high estimation noise.
arXiv Detail & Related papers (2021-10-01T17:50:46Z) - LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial
Attack [74.5144793386864]
LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input sample and that of an adversarial sample.
LSD works directly in the image pixel domain to guarantee that non-$ell$ constraints, such as sparsity, are satisfied.
arXiv Detail & Related papers (2021-03-19T13:10:47Z) - A Linearly Convergent Algorithm for Distributed Principal Component
Analysis [12.91948651812873]
This paper introduces a feedforward neural network-based one time-scale distributed PCA algorithm termed Distributed Sanger's Algorithm (DSA)
The proposed algorithm is shown to converge linearly to a neighborhood of the true solution.
arXiv Detail & Related papers (2021-01-05T00:51:14Z) - Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature
Selection [138.97647716793333]
We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization.
We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
arXiv Detail & Related papers (2020-12-29T04:08:38Z) - Sparse Feature Selection Makes Batch Reinforcement Learning More Sample
Efficient [62.24615324523435]
This paper provides a statistical analysis of high-dimensional batch Reinforcement Learning (RL) using sparse linear function approximation.
When there is a large number of candidate features, our result sheds light on the fact that sparsity-aware methods can make batch RL more sample efficient.
arXiv Detail & Related papers (2020-11-08T16:48:02Z) - PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions [71.60219086238254]
We deal with the issue from the connection between convolutions and partial differential operators (PDOs)
In implementation, we discretize the system using the numerical schemes of PDOs, deriving approximately equivariant convolutions (PDO-eConvs)
Experiments on rotated MNIST and natural image classification show that PDO-eConvs perform competitively yet use parameters much more efficiently.
arXiv Detail & Related papers (2020-07-20T18:57:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.