Methods for Class-Imbalanced Learning with Support Vector Machines: A Review and an Empirical Evaluation

• URL: http://arxiv.org/abs/2406.03398v2
• Date: Wed, 12 Jun 2024 02:37:08 GMT
• Title: Methods for Class-Imbalanced Learning with Support Vector Machines: A Review and an Empirical Evaluation
• Authors: Salim Rezvani, Farhad Pourpanah, Chee Peng Lim, Q. M. Jonathan Wu,
• Abstract summary: We introduce a hierarchical categorization of SVM-based models with respect to class-imbalanced learning. We compare the performances of various representative SVM-based models in each category using benchmark imbalanced data sets. Our findings reveal that while algorithmic methods are less time-consuming owing to no data pre-processing requirements, fusion methods, which combine both re-sampling and algorithmic approaches, generally perform the best.
• Score: 22.12895887111828
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This paper presents a review on methods for class-imbalanced learning with the Support Vector Machine (SVM) and its variants. We first explain the structure of SVM and its variants and discuss their inefficiency in learning with class-imbalanced data sets. We introduce a hierarchical categorization of SVM-based models with respect to class-imbalanced learning. Specifically, we categorize SVM-based models into re-sampling, algorithmic, and fusion methods, and discuss the principles of the representative models in each category. In addition, we conduct a series of empirical evaluations to compare the performances of various representative SVM-based models in each category using benchmark imbalanced data sets, ranging from low to high imbalanced ratios. Our findings reveal that while algorithmic methods are less time-consuming owing to no data pre-processing requirements, fusion methods, which combine both re-sampling and algorithmic approaches, generally perform the best, but with a higher computational load. A discussion on research gaps and future research directions is provided.

Related papers

• Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification [75.90957645766676]
  We present a novel approach that combines the eigenanalysis of a covariance matrix evaluated on a training set with a Hessian matrix evaluated on a deep learning model. Our approach is substantiated by formal proofs that establish its capability to maximize between-class mean distance and minimize within-class variances.
  arXiv  Detail & Related papers  (2024-02-14T16:10:42Z)
• Federated Learning Aggregation: New Robust Algorithms with Guarantees [63.96013144017572]
  Federated learning has been recently proposed for distributed model training at the edge. This paper presents a complete general mathematical convergence analysis to evaluate aggregation strategies in a federated learning framework. We derive novel aggregation algorithms which are able to modify their model architecture by differentiating client contributions according to the value of their losses.
  arXiv  Detail & Related papers  (2022-05-22T16:37:53Z)
• Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization [73.17488635491262]
  Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs.
  arXiv  Detail & Related papers  (2022-04-21T16:08:44Z)
• Using Explainable Boosting Machine to Compare Idiographic and Nomothetic Approaches for Ecological Momentary Assessment Data [2.0824228840987447]
  This paper explores the use of non-linear interpretable machine learning (ML) models in classification problems. Various ensembles of trees are compared to linear models using imbalanced synthetic and real-world datasets. In one of the two real-world datasets, knowledge distillation method achieves improved AUC scores.
  arXiv  Detail & Related papers  (2022-04-04T17:56:37Z)
• Self-Supervised Class Incremental Learning [51.62542103481908]
  Existing Class Incremental Learning (CIL) methods are based on a supervised classification framework sensitive to data labels. When updating them based on the new class data, they suffer from catastrophic forgetting: the model cannot discern old class data clearly from the new. In this paper, we explore the performance of Self-Supervised representation learning in Class Incremental Learning (SSCIL) for the first time.
  arXiv  Detail & Related papers  (2021-11-18T06:58:19Z)
• SetConv: A New Approach for Learning from Imbalanced Data [29.366843553056594]
  We propose a set convolution operation and an episodic training strategy to extract a single representative for each class. We prove that our proposed algorithm is permutation-invariant despite the order of inputs.
  arXiv  Detail & Related papers  (2021-04-03T22:33:30Z)
• Estimating Average Treatment Effects with Support Vector Machines [77.34726150561087]
  Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature. We adapt SVM as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups. We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods.
  arXiv  Detail & Related papers  (2021-02-23T20:22:56Z)
• Unsupervised Real Time Prediction of Faults Using the Support Vector Machine [1.1852751647387592]
  We show that the proposed solution can perform much better when using the SMO training algorithm. The classification performance of this predictive model is considerably better than the SVM with and without SMO training algorithm.
  arXiv  Detail & Related papers  (2020-12-30T04:27:10Z)
• Equipment Failure Analysis for Oil and Gas Industry with an Ensemble Predictive Model [0.0]
  We show that the proposed solution can perform much better when using the SMO training algorithm. The classification performance of this predictive model is considerably better than the SVM with and without SMO training algorithm.
  arXiv  Detail & Related papers  (2020-12-30T04:14:15Z)
• A Mathematical Programming approach to Binary Supervised Classification with Label Noise [1.2031796234206138]
  We propose novel methodologies to construct Support Vector Machine -based classifiers. The first method incorporates relabeling directly in the SVM model. A second family of methods combines clustering with classification at the same time, giving rise to a model that applies simultaneously similarity measures and SVM.
  arXiv  Detail & Related papers  (2020-04-21T17:25:54Z)
• Learning to Select Base Classes for Few-shot Classification [96.92372639495551]
  We use the Similarity Ratio as an indicator for the generalization performance of a few-shot model. We then formulate the base class selection problem as a submodular optimization problem over Similarity Ratio.
  arXiv  Detail & Related papers  (2020-04-01T09:55:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.