Feature Selection using Sparse Adaptive Bottleneck Centroid-Encoder

• URL: http://arxiv.org/abs/2306.04795v2
• Date: Fri, 9 Jun 2023 03:56:10 GMT
• Title: Feature Selection using Sparse Adaptive Bottleneck Centroid-Encoder
• Authors: Tomojit Ghosh, Michael Kirby
• Abstract summary: We introduce a novel nonlinear model, Sparse Adaptive Bottleneckid-Encoder (SABCE), for determining the features that discriminate between two or more classes. The algorithm is applied to various real-world data sets, including high-dimensional biological, image, speech, and accelerometer sensor data.
• Score: 1.2487990897680423
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a novel nonlinear model, Sparse Adaptive Bottleneck Centroid-Encoder (SABCE), for determining the features that discriminate between two or more classes. The algorithm aims to extract discriminatory features in groups while reconstructing the class centroids in the ambient space and simultaneously use additional penalty terms in the bottleneck layer to decrease within-class scatter and increase the separation of different class centroids. The model has a sparsity-promoting layer (SPL) with a one-to-one connection to the input layer. Along with the primary objective, we minimize the $l_{2,1}$-norm of the sparse layer, which filters out unnecessary features from input data. During training, we update class centroids by taking the Hadamard product of the centroids and weights of the sparse layer, thus ignoring the irrelevant features from the target. Therefore the proposed method learns to reconstruct the critical components of class centroids rather than the whole centroids. The algorithm is applied to various real-world data sets, including high-dimensional biological, image, speech, and accelerometer sensor data. We compared our method to different state-of-the-art feature selection techniques, including supervised Concrete Autoencoders (SCAE), Feature Selection Networks (FsNet), Stochastic Gates (STG), and LassoNet. We empirically showed that SABCE features often produced better classification accuracy than other methods on the sequester test sets, setting new state-of-the-art results.

Related papers

• Prototypical Hash Encoding for On-the-Fly Fine-Grained Category Discovery [65.16724941038052]
  Category-aware Prototype Generation (CPG) and Discrimi Category 5.3% (DCE) are proposed. CPG enables the model to fully capture the intra-category diversity by representing each category with multiple prototypes. DCE boosts the discrimination ability of hash code with the guidance of the generated category prototypes.
  arXiv  Detail & Related papers  (2024-10-24T23:51:40Z)
• Exploration of Class Center for Fine-Grained Visual Classification [7.120809788357261]
  Fine-grained visual classification is a challenging task due to intra-class variances and subtle inter-class differences. We propose a loss function named exploration of class center, which consists of a multiple class-center constraint and a class-center label generation. Our method can be easily integrated with existing fine-grained visual classification approaches as a loss function, to further boost excellent performance with only slight training costs.
  arXiv  Detail & Related papers  (2024-07-05T04:11:09Z)
• Rethinking Few-shot 3D Point Cloud Semantic Segmentation [62.80639841429669]
  This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS) We focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution. To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built.
  arXiv  Detail & Related papers  (2024-03-01T15:14:47Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• A Weighted K-Center Algorithm for Data Subset Selection [70.49696246526199]
  Subset selection is a fundamental problem that can play a key role in identifying smaller portions of the training data. We develop a novel factor 3-approximation algorithm to compute subsets based on the weighted sum of both k-center and uncertainty sampling objective functions.
  arXiv  Detail & Related papers  (2023-12-17T04:41:07Z)
• DiGeo: Discriminative Geometry-Aware Learning for Generalized Few-Shot Object Detection [39.937724871284665]
  Generalized few-shot object detection aims to achieve precise detection on both base classes with abundant annotations and novel classes with limited training data. Existing approaches enhance few-shot generalization with the sacrifice of base-class performance. We propose a new training framework, DiGeo, to learn Geometry-aware features of inter-class separation and intra-class compactness.
  arXiv  Detail & Related papers  (2023-03-16T22:37:09Z)
• Sparse Centroid-Encoder: A Nonlinear Model for Feature Selection [1.2487990897680423]
  We develop a sparse implementation of the centroid-encoder for nonlinear data reduction and visualization called Centro Sparseid-Encoder. We also provide a feature selection framework that first ranks each feature by its occurrence, and the optimal number of features is chosen using a validation set. The algorithm is applied to a wide variety of data sets including, single-cell biological data, high dimensional infectious disease data, hyperspectral data, image data, and speech data.
  arXiv  Detail & Related papers  (2022-01-30T20:46:24Z)
• A SPA-based Manifold Learning Framework for Motor Imagery EEG Data Classification [2.4727719996518487]
  This paper proposes a manifold learning framework to classify two types of EEG data from motor imagery (MI) tasks. For feature extraction, it is implemented by Common Spatial Pattern (CSP) from the preprocessed EEG signals. In the neighborhoods of the features for classification, the local approximation to the support of the data is obtained, and then the features are assigned to the classes with the closest support.
  arXiv  Detail & Related papers  (2021-07-30T06:18:05Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• Learning Class Regularized Features for Action Recognition [68.90994813947405]
  We introduce a novel method named Class Regularization that performs class-based regularization of layer activations. We show that using Class Regularization blocks in state-of-the-art CNN architectures for action recognition leads to systematic improvement gains of 1.8%, 1.2% and 1.4% on the Kinetics, UCF-101 and HMDB-51 datasets, respectively.
  arXiv  Detail & Related papers  (2020-02-07T07:27:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.