Challenges in Binary Classification

• URL: http://arxiv.org/abs/2406.13665v1
• Date: Wed, 19 Jun 2024 16:11:59 GMT
• Title: Challenges in Binary Classification
• Authors: Pengbo Yang, Jian Yu,
• Abstract summary: The selection of kernel function is empirical, which means that the kernel function may not be optimal. For linear classification, it can be deduced that SVM is a special case of this variational problem framework. For Euclidean distance, it is proved that the proposed variational problem has some limitations for nonlinear classification.
• Score: 12.827730806517664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Binary Classification plays an important role in machine learning. For linear classification, SVM is the optimal binary classification method. For nonlinear classification, the SVM algorithm needs to complete the classification task by using the kernel function. Although the SVM algorithm with kernel function is very effective, the selection of kernel function is empirical, which means that the kernel function may not be optimal. Therefore, it is worth studying how to obtain an optimal binary classifier. In this paper, the problem of finding the optimal binary classifier is considered as a variational problem. We design the objective function of this variational problem through the max-min problem of the (Euclidean) distance between two classes. For linear classification, it can be deduced that SVM is a special case of this variational problem framework. For Euclidean distance, it is proved that the proposed variational problem has some limitations for nonlinear classification. Therefore, how to design a more appropriate objective function to find the optimal binary classifier is still an open problem. Further, it's discussed some challenges and problems in finding the optimal classifier.

Related papers

• Data Selection for ERMs [67.57726352698933]
  We study how well can $mathcalA$ perform when trained on at most $nll N$ data points selected from a population of $N$ points. Our results include optimal data-selection bounds for mean estimation, linear classification, and linear regression.
  arXiv  Detail & Related papers  (2025-04-20T11:26:01Z)
• Multi-Class Imbalanced Learning with Support Vector Machines via Differential Evolution [4.877822002065298]
  Support vector machine (SVM) is a powerful machine learning algorithm to handle classification tasks. In this paper, we propose an improved SVM via Differential Evolution (i-SVM-DE) method to deal with it.
  arXiv  Detail & Related papers  (2025-02-20T14:30:18Z)
• Efficient Approximate Kernel Based Spike Sequence Classification [56.2938724367661]
  Machine learning models, such as SVM, require a definition of distance/similarity between pairs of sequences. Exact methods yield better classification performance, but they pose high computational costs. We propose a series of ways to improve the performance of the approximate kernel in order to enhance its predictive performance.
  arXiv  Detail & Related papers  (2022-09-11T22:44:19Z)
• Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization [73.17488635491262]
  Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs.
  arXiv  Detail & Related papers  (2022-04-21T16:08:44Z)
• On the Benefits of Large Learning Rates for Kernel Methods [110.03020563291788]
  We show that a phenomenon can be precisely characterized in the context of kernel methods. We consider the minimization of a quadratic objective in a separable Hilbert space, and show that with early stopping, the choice of learning rate influences the spectral decomposition of the obtained solution.
  arXiv  Detail & Related papers  (2022-02-28T13:01:04Z)
• Implicit differentiation for fast hyperparameter selection in non-smooth convex learning [87.60600646105696]
  We study first-order methods when the inner optimization problem is convex but non-smooth. We show that the forward-mode differentiation of proximal gradient descent and proximal coordinate descent yield sequences of Jacobians converging toward the exact Jacobian.
  arXiv  Detail & Related papers  (2021-05-04T17:31:28Z)
• Theoretical Insights Into Multiclass Classification: A High-dimensional Asymptotic View [82.80085730891126]
  We provide the first modernally precise analysis of linear multiclass classification. Our analysis reveals that the classification accuracy is highly distribution-dependent. The insights gained may pave the way for a precise understanding of other classification algorithms.
  arXiv  Detail & Related papers  (2020-11-16T05:17:29Z)
• A fast learning algorithm for One-Class Slab Support Vector Machines [1.1613446814180841]
  This paper proposes fast training method for One Class Slab SVMs using an updated Sequential Minimal Optimization (SMO) The results indicate that this training method scales better to large sets of training data than other Quadratic Programming (QP) solvers.
  arXiv  Detail & Related papers  (2020-11-06T09:16:39Z)
• Robust Multi-class Feature Selection via $l_{2,0}$-Norm Regularization Minimization [6.41804410246642]
  Feature selection is an important computational-processing in data mining and machine learning. In this paper, a novel method based on homoy hard threshold (HIHT) is proposed to solve the least square problem for multi-class feature selection.
  arXiv  Detail & Related papers  (2020-10-08T02:06:06Z)
• Self-Weighted Robust LDA for Multiclass Classification with Edge Classes [111.5515086563592]
  A novel self-weighted robust LDA with l21-norm based between-class distance criterion, called SWRLDA, is proposed for multi-class classification. The proposed SWRLDA is easy to implement, and converges fast in practice.
  arXiv  Detail & Related papers  (2020-09-24T12:32:55Z)
• Frank-Wolfe algorithm for learning SVM-type multi-category classifiers [0.7646713951724012]
  Multi-category support vector machine (MC-SVM) is one of the most popular machine learning algorithms. In this study, we developed a new optimization algorithm that can be applied to many of MC-SVM variants.
  arXiv  Detail & Related papers  (2020-08-20T11:19:07Z)
• A novel embedded min-max approach for feature selection in nonlinear support vector machine classification [0.0]
  We propose an embedded feature selection method based on a min-max optimization problem. By leveraging duality theory, we equivalently reformulate the min-max problem and solve it without further ado. The efficiency and usefulness of our approach are tested on several benchmark data sets.
  arXiv  Detail & Related papers  (2020-04-21T09:40:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.