Local Duality for Sparse Support Vector Machines

• URL: http://arxiv.org/abs/2601.20170v1
• Date: Wed, 28 Jan 2026 02:09:52 GMT
• Title: Local Duality for Sparse Support Vector Machines
• Authors: Penghe Zhang, Naihua Xiu, Houduo Qi,
• Abstract summary: sparse support vector machines (SSVMs) have attracted much attention lately and show certain empirical advantages over convex SVMs.<n>This paper develops a local duality theory for such an SSVM formulation and explores its relationship with the hinge-loss SVM and the ramp-loss SVM.
• Score: 3.562094249178102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the rise of cardinality minimization in optimization, sparse support vector machines (SSVMs) have attracted much attention lately and show certain empirical advantages over convex SVMs. A common way to derive an SSVM is to add a cardinality function such as $\ell_0$-norm to the dual problem of a convex SVM. However, this process lacks theoretical justification. This paper fills the gap by developing a local duality theory for such an SSVM formulation and exploring its relationship with the hinge-loss SVM (hSVM) and the ramp-loss SVM (rSVM). In particular, we prove that the derived SSVM is exactly the dual problem of the 0/1-loss SVM, and the linear representer theorem holds for their local solutions. The local solution of SSVM also provides guidelines on selecting hyperparameters of hSVM and rSVM. {Under specific conditions, we show that a sequence of global solutions of hSVM converges to a local solution of 0/1-loss SVM. Moreover, a local minimizer of 0/1-loss SVM is a local minimizer of rSVM.} This explains why a local solution induced by SSVM outperforms hSVM and rSVM in the prior empirical study. We further conduct numerical tests on real datasets and demonstrate potential advantages of SSVM by working with locally nice solutions proposed in this paper.

Related papers

• From Images to Signals: Are Large Vision Models Useful for Time Series Analysis? [62.58235852194057]
  Transformer-based models have gained increasing attention in time series research.<n>As the field moves toward multi-modality, Large Vision Models (LVMs) are emerging as a promising direction.
  arXiv  Detail & Related papers  (2025-05-29T22:05:28Z)
• $p$SVM: Soft-margin SVMs with $p$-norm Hinge Loss [0.0]
  Support Vector Machines (SVMs) based on hinge loss have been extensively discussed and applied to various binary classification tasks. In this paper, we explore the properties, performance, and training algorithms of $p$SVMs.
  arXiv  Detail & Related papers  (2024-08-19T11:30:00Z)
• Multi-class Support Vector Machine with Maximizing Minimum Margin [60.06805919852749]
  Support Vector Machine (SVM) is a prominent machine learning technique widely applied in pattern recognition tasks.<n>We propose a novel method for multi-class SVM that incorporates pairwise class loss considerations and maximizes the minimum margin.<n> Empirical evaluations demonstrate the effectiveness and superiority of our proposed method over existing multi-classification methods.
  arXiv  Detail & Related papers  (2023-12-11T18:09:55Z)
• Low-Rank Multitask Learning based on Tensorized SVMs and LSSVMs [65.42104819071444]
  Multitask learning (MTL) leverages task-relatedness to enhance performance. We employ high-order tensors, with each mode corresponding to a task index, to naturally represent tasks referenced by multiple indices. We propose a general framework of low-rank MTL methods with tensorized support vector machines (SVMs) and least square support vector machines (LSSVMs)
  arXiv  Detail & Related papers  (2023-08-30T14:28:26Z)
• Lp- and Risk Consistency of Localized SVMs [0.0]
  Kernel-based regularized risk minimizers, also called support vector machines (SVMs), are known to possess many desirable properties but suffer from their super-linear computational requirements when dealing with large data sets. In this paper, localized SVMs are analyzed with regards to their consistency.
  arXiv  Detail & Related papers  (2023-05-16T12:11:08Z)
• New Equivalences Between Interpolation and SVMs: Kernels and Structured Features [22.231455330003328]
  We present a new and flexible analysis framework for proving SVP in an arbitrary kernel reproducing Hilbert space with a flexible class of generative models for the labels. We show that SVP occurs in many interesting settings not covered by prior work, and we leverage these results to prove novel generalization results for kernel SVM classification.
  arXiv  Detail & Related papers  (2023-05-03T17:52:40Z)
• Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization [73.17488635491262]
  Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs.
  arXiv  Detail & Related papers  (2022-04-21T16:08:44Z)
• Chance constrained conic-segmentation support vector machine with uncertain data [0.0]
  Support vector machines (SVM) is one of the well known supervised classes of learning algorithms. This paper studies CS-SVM when the data points are uncertain or mislabelled.
  arXiv  Detail & Related papers  (2021-07-28T12:29:47Z)
• Machine-Learning-Derived Entanglement Witnesses [55.76279816849472]
  We show a correspondence between linear support vector machines (SVMs) and entanglement witnesses. We use this correspondence to generate entanglement witnesses for bipartite and tripartite qubit (and qudit) target entangled states.
  arXiv  Detail & Related papers  (2021-07-05T22:28:02Z)
• Estimating Average Treatment Effects with Support Vector Machines [77.34726150561087]
  Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature. We adapt SVM as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups. We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods.
  arXiv  Detail & Related papers  (2021-02-23T20:22:56Z)
• Learning a powerful SVM using piece-wise linear loss functions [0.0]
  k-Piece-wise Linear loss Support Vector Machine (k-PL-SVM) model is an adaptive SVM model. We have performed the extensive numerical experiments with k-PL-SVM models for k = 2 and 3.
  arXiv  Detail & Related papers  (2021-02-09T14:45:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.