Bi-Sparse Unsupervised Feature Selection

• URL: http://arxiv.org/abs/2412.16819v2
• Date: Thu, 14 Aug 2025 01:54:33 GMT
• Title: Bi-Sparse Unsupervised Feature Selection
• Authors: Xianchao Xiu, Chenyi Huang, Pan Shang, Wanquan Liu,
• Abstract summary: We propose a unified framework for bi-sparse PCA analysis, called BSUFS.<n>In this paper, we show the effectiveness of our proposed BSUFS on synthetic and real-world datasets.<n>The results reveal the advantages of bi-sparse optimization in selection and show its potential for other fields in image processing.
• Score: 9.541908550559361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To deal with high-dimensional unlabeled datasets in many areas, principal component analysis (PCA) has become a rising technique for unsupervised feature selection (UFS). However, most existing PCA-based methods only consider the structure of datasets by embedding a single sparse regularization or constraint on the transformation matrix. In this paper, we introduce a novel bi-sparse method called BSUFS to improve the performance of UFS. The core idea of BSUFS is to incorporate $\ell_{2,p}$-norm and $\ell_q$-norm into the classical PCA, which enables our method to select relevant features and filter out irrelevant noises, thereby obtaining discriminative features. Here, the parameters $p$ and $q$ are within the range of $[0, 1)$. Therefore, BSUFS not only constructs a unified framework for bi-sparse optimization, but also includes some existing works as special cases. To solve the resulting non-convex model, we propose an efficient proximal alternating minimization (PAM) algorithm using Stiefel manifold optimization and sparse optimization techniques. In addition, the computational complexity analysis is presented. Extensive numerical experiments on synthetic and real-world datasets demonstrate the effectiveness of our proposed BSUFS. The results reveal the advantages of bi-sparse optimization in feature selection and show its potential for other fields in image processing. Our code is available at https://github.com/xianchaoxiu/BSUFS.

Related papers

• Closing the Approximation Gap of Partial AUC Optimization: A Tale of Two Formulations [121.39938773554523]
  The Area Under the ROC Curve (AUC) is a pivotal evaluation metric in real-world scenarios with both class imbalance and decision constraints.<n>We present two simple instance-wise minimax reformulations to close the approximation gap of PAUC optimization.<n>The resulting algorithms enjoy a linear per-iteration computational complexity w.r.t. the sample size and a convergence rate of $O(-2/3)$ for typical one-way and two-way PAUCs.
  arXiv  Detail & Related papers  (2025-12-01T02:52:33Z)
• Bi-Level Unsupervised Feature Selection [11.383408944117804]
  We propose a novel bi-level unsupervised feature selection (BLUFS) method, including a clustering level and a feature level.<n>At the clustering level, spectral clustering is used to generate pseudo-labels for representing the data structure, while a continuous linear regression model is developed to learn the projection matrix.<n>At the feature level, the $ell_2,0$-norm constraint is imposed on the projection matrix for more effectively selecting features.
  arXiv  Detail & Related papers  (2025-05-26T22:52:31Z)
• Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization [6.342485512772862]
  Unsupervised single feature selection (UFS) is widely applied in machine learning and pattern recognition. Most of the existing methods only consider sparsity, which makes it difficult to select subsets and discriminative from the original. In this paper, we propose a new method called DSCOFS to consider a subset and discriminative from the original.
  arXiv  Detail & Related papers  (2025-01-01T05:05:46Z)
• Optimizing Posterior Samples for Bayesian Optimization via Rootfinding [2.94944680995069]
  We introduce an efficient global optimization strategy for posterior samples based on global rootfinding.<n>Remarkably, even with just one point from each set, the global optimum is discovered most of the time.<n>Our approach also improves the performance of other posterior sample-based acquisition functions, such as variants of entropy search.
  arXiv  Detail & Related papers  (2024-10-29T17:57:16Z)
• Optimal Bound for PCA with Outliers using Higher-Degree Voronoi Diagrams [0.0]
  We introduce new algorithms for Principal Component Analysis (PCA) with outliers.<n>We navigate to the optimal subspace for PCA even in the presence of outliers.<n>This approach achieves an optimal solution with a time complexity of $nd+mathcalO(1)textpoly(n,d)$.
  arXiv  Detail & Related papers  (2024-08-13T13:05:36Z)
• Memory-Efficient Gradient Unrolling for Large-Scale Bi-level Optimization [71.35604981129838]
  Bi-level optimization has become a fundamental mathematical framework for addressing hierarchical machine learning problems.<n>Traditional gradient-based bi-level optimization algorithms are ill-suited to meet the demands of large-scale applications.<n>We introduce $(textFG)2textU$, which achieves an unbiased approximation of the meta gradient for bi-level optimization.
  arXiv  Detail & Related papers  (2024-06-20T08:21:52Z)
• Poisson Process for Bayesian Optimization [126.51200593377739]
  We propose a ranking-based surrogate model based on the Poisson process and introduce an efficient BO framework, namely Poisson Process Bayesian Optimization (PoPBO) Compared to the classic GP-BO method, our PoPBO has lower costs and better robustness to noise, which is verified by abundant experiments.
  arXiv  Detail & Related papers  (2024-02-05T02:54:50Z)
• DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning [52.83811558753284]
  Decentralized learning (DFL) discards the central server and establishes a decentralized communication network. Existing DFL methods still suffer from two major challenges: local inconsistency and local overfitting.
  arXiv  Detail & Related papers  (2023-08-16T11:22:36Z)
• Learning Regions of Interest for Bayesian Optimization with Adaptive Level-Set Estimation [84.0621253654014]
  We propose a framework, called BALLET, which adaptively filters for a high-confidence region of interest. We show theoretically that BALLET can efficiently shrink the search space, and can exhibit a tighter regret bound than standard BO.
  arXiv  Detail & Related papers  (2023-07-25T09:45:47Z)
• Sparse Linear Centroid-Encoder: A Convex Method for Feature Selection [1.057079240576682]
  We present Sparse Centroid-Encoder (SLCE) over a novel feature selection technique. The algorithm uses a linear network to reconstruct a neural feature at the same time.
  arXiv  Detail & Related papers  (2023-06-07T23:07:55Z)
• Beyond Discrete Selection: Continuous Embedding Space Optimization for Generative Feature Selection [34.32619834917906]
  We reformulate the feature selection problem as a deep differentiable optimization task. We propose a new principled research perspective: conceptualizing discrete feature subsetting as continuous embedding space. Specifically, we utilize reinforcement feature selection learning to generate diverse and high-quality training data.
  arXiv  Detail & Related papers  (2023-02-26T03:18:45Z)
• Large-scale Optimization of Partial AUC in a Range of False Positive Rates [51.12047280149546]
  The area under the ROC curve (AUC) is one of the most widely used performance measures for classification models in machine learning. We develop an efficient approximated gradient descent method based on recent practical envelope smoothing technique. Our proposed algorithm can also be used to minimize the sum of some ranked range loss, which also lacks efficient solvers.
  arXiv  Detail & Related papers  (2022-03-03T03:46:18Z)
• Fully Convolutional Networks for Panoptic Segmentation with Point-based Supervision [88.71403886207071]
  We present a conceptually simple, strong, and efficient framework for fully- and weakly-supervised panoptic segmentation, called Panoptic FCN. Our approach aims to represent and predict foreground things and background stuff in a unified fully convolutional pipeline. Panoptic FCN encodes each object instance or stuff category with the proposed kernel generator and produces the prediction by convolving the high-resolution feature directly.
  arXiv  Detail & Related papers  (2021-08-17T15:28:53Z)
• An Online Riemannian PCA for Stochastic Canonical Correlation Analysis [37.8212762083567]
  We present an efficient algorithm (RSG+) for canonical correlation analysis (CCA) using a reparametrization of the projection matrices. While the paper primarily focuses on the formulation and technical analysis of its properties, our experiments show that the empirical behavior on common datasets is quite promising.
  arXiv  Detail & Related papers  (2021-06-08T23:38:29Z)
• Bayesian Optimistic Optimisation with Exponentially Decaying Regret [58.02542541410322]
  The current practical BO algorithms have regret bounds ranging from $mathcalO(fraclogNsqrtN)$ to $mathcal O(e-sqrtN)$, where $N$ is the number of evaluations. This paper explores the possibility of improving the regret bound in the noiseless setting by intertwining concepts from BO and tree-based optimistic optimisation. We propose the BOO algorithm, a first practical approach which can achieve an exponential regret bound with order $mathcal O(N-sqrt
  arXiv  Detail & Related papers  (2021-05-10T13:07:44Z)
• Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information [78.78486761923855]
  In many real world problems, we want to infer some property of an expensive black-box function f, given a budget of T function evaluations. We present a procedure, InfoBAX, that sequentially chooses queries that maximize mutual information with respect to the algorithm's output. On these problems, InfoBAX uses up to 500 times fewer queries to f than required by the original algorithm.
  arXiv  Detail & Related papers  (2021-04-19T17:22:11Z)
• Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence [66.83161885378192]
  Area under ROC (AUROC) and precision-recall curves (AUPRC) are common metrics for evaluating classification performance for imbalanced problems. We propose a technical method to optimize AUPRC for deep learning.
  arXiv  Detail & Related papers  (2021-04-18T06:22:21Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)
• FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data [59.50904660420082]
  Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model.
  arXiv  Detail & Related papers  (2020-05-22T23:07:42Z)
• Practical Bayesian Optimization of Objectives with Conditioning Variables [1.0497128347190048]
  We consider the more general case where a user is faced with multiple problems that each need to be optimized conditional on a state variable. Similarity across objectives boosts optimization of each objective in two ways. We propose a framework for conditional optimization: ConBO.
  arXiv  Detail & Related papers  (2020-02-23T22:06:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.