Multi-Class Support Vector Machine with Differential Privacy

• URL: http://arxiv.org/abs/2510.04027v1
• Date: Sun, 05 Oct 2025 04:25:16 GMT
• Title: Multi-Class Support Vector Machine with Differential Privacy
• Authors: Jinseong Park, Yujin Choi, Jaewook Lee,
• Abstract summary: differential privacy is one of the major frameworks to build privacy-preserving machine learning models.<n>Applying DP to multi-class SVMs is inadequate, as the standard one-versus-rest (OvR) and one-versus-one (OvO) approaches repeatedly query each data sample.<n>We propose a novel differentially Private Multi-class SVM (PMSVM) with weight and gradient perturbation methods to ensure DP in all-in-one SVMs.
• Score: 12.305474267080966
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the increasing need to safeguard data privacy in machine learning models, differential privacy (DP) is one of the major frameworks to build privacy-preserving models. Support Vector Machines (SVMs) are widely used traditional machine learning models due to their robust margin guarantees and strong empirical performance in binary classification. However, applying DP to multi-class SVMs is inadequate, as the standard one-versus-rest (OvR) and one-versus-one (OvO) approaches repeatedly query each data sample when building multiple binary classifiers, thus consuming the privacy budget proportionally to the number of classes. To overcome this limitation, we explore all-in-one SVM approaches for DP, which access each data sample only once to construct multi-class SVM boundaries with margin maximization properties. We propose a novel differentially Private Multi-class SVM (PMSVM) with weight and gradient perturbation methods, providing rigorous sensitivity and convergence analyses to ensure DP in all-in-one SVMs. Empirical results demonstrate that our approach surpasses existing DP-SVM methods in multi-class scenarios.

Related papers

• From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation [59.27094165576015]
  We propose a novel learning paradigm (UniMod) that transitions from sparse decision-making to dense reasoning traces.<n>By constructing structured trajectories encompassing evidence grounding, modality assessment, risk mapping, policy decision, and response generation, we reformulate monolithic decision tasks into a multi-dimensional boundary learning process.<n>We introduce specialized optimization strategies to decouple task-specific parameters and rebalance training dynamics, effectively resolving interference between diverse objectives in multi-task learning.
  arXiv  Detail & Related papers  (2026-01-28T09:29:40Z)
• Twin Restricted Kernel Machines for Multiview Classification [10.208788616684162]
  Multi-view learning (MVL) focuses on improving generalization performance by leveraging complementary information from multiple perspectives or views.<n>We introduce the multiview twin restricted kernel machine (TMvRKM), a novel model that integrates the strengths of kernel machines with the multiview framework.<n>The proposed TMvRKM model is rigorously tested on UCI, KEEL, and AwA benchmark datasets.
  arXiv  Detail & Related papers  (2025-12-12T03:54:19Z)
• Adaptive Weighted LSSVM for Multi-View Classification [0.5161531917413708]
  AW-LSSVM promotes complementary learning by an iterative global coupling to make each view focus on hard samples of others from previous iterations.<n>Experiments demonstrate AW-LSSVM outperforms existing kernel-based multi-view methods on most datasets.
  arXiv  Detail & Related papers  (2025-12-02T11:14:47Z)
• Hierarchical Identity Learning for Unsupervised Visible-Infrared Person Re-Identification [81.3063589622217]
  Unsupervised visible-infrared person re-identification (USVI-ReID) aims to learn modality-invariant image features from unlabeled cross-modal person datasets.
  arXiv  Detail & Related papers  (2025-09-15T05:10:43Z)
• MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings [75.0617088717528]
  MoCa is a framework for transforming pre-trained VLM backbones into effective bidirectional embedding models.<n>MoCa consistently improves performance across MMEB and ViDoRe-v2 benchmarks, achieving new state-of-the-art results.
  arXiv  Detail & Related papers  (2025-06-29T06:41:00Z)
• Robust Multi-View Learning via Representation Fusion of Sample-Level Attention and Alignment of Simulated Perturbation [61.64052577026623]
  Real-world multi-view datasets are often heterogeneous and imperfect.<n>We propose a novel robust MVL method (namely RML) with simultaneous representation fusion and alignment.<n>Our RML is self-supervised and can also be applied for downstream tasks as a regularization.
  arXiv  Detail & Related papers  (2025-03-06T07:01:08Z)
• Multiview learning with twin parametric margin SVM [0.0]
  Multiview learning (MVL) seeks to leverage the benefits of diverse perspectives to complement each other. We propose multiview twin parametric margin support vector machine (MvTPMSVM) MvTPMSVM constructs parametric margin hyperplanes corresponding to both classes, aiming to regulate and manage the impact of the heteroscedastic noise structure.
  arXiv  Detail & Related papers  (2024-08-04T10:16:11Z)
• Multi-class Support Vector Machine with Maximizing Minimum Margin [60.06805919852749]
  Support Vector Machine (SVM) is a prominent machine learning technique widely applied in pattern recognition tasks.<n>We propose a novel method for multi-class SVM that incorporates pairwise class loss considerations and maximizes the minimum margin.<n> Empirical evaluations demonstrate the effectiveness and superiority of our proposed method over existing multi-classification methods.
  arXiv  Detail & Related papers  (2023-12-11T18:09:55Z)
• Enhancing Pattern Classification in Support Vector Machines through Matrix Formulation [0.0]
  The reliance on vector-based formulations in existing SVM-based models poses limitations regarding flexibility and ease of incorporating additional terms to handle specific challenges. We introduce a matrix formulation for SVM that effectively addresses these constraints. Experimental evaluations on multilabel and multiclass datasets demonstrate that Matrix SVM achieves superior time efficacy.
  arXiv  Detail & Related papers  (2023-07-18T15:56:39Z)
• A Robust Twin Parametric Margin Support Vector Machine for Multiclass Classification [0.0]
  We introduce novel Twin Parametric Margin Support Vector Machine (TPMSVM) models designed to address multiclass classification tasks under feature uncertainty.<n>To handle data perturbations, we construct bounded-by-norm uncertainty set around each training observation and derive the robust counterparts of the deterministic models.<n>We validate the effectiveness of the proposed robust multiclass TPMSVM methodology on real-world datasets.
  arXiv  Detail & Related papers  (2023-06-09T19:27:24Z)
• Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information [77.80071279597665]
  We propose an all-in-one single-stage pre-training approach, named Maximizing Multi-modal Mutual Information Pre-training (M3I Pre-training) Our approach achieves better performance than previous pre-training methods on various vision benchmarks, including ImageNet classification, object detection, LVIS long-tailed object detection, and ADE20k semantic segmentation.
  arXiv  Detail & Related papers  (2022-11-17T18:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.