Sparse Universum Quadratic Surface Support Vector Machine Models for Binary Classification

• URL: http://arxiv.org/abs/2104.01331v1
• Date: Sat, 3 Apr 2021 07:40:30 GMT
• Title: Sparse Universum Quadratic Surface Support Vector Machine Models for Binary Classification
• Authors: Hossein Moosaei, Ahmad Mousavi, Milan Hlad\'ik, Zheming Gao
• Abstract summary: We design kernel-free Universum quadratic surface support vector machine models. We propose the L1 norm regularized version that is beneficial for detecting potential sparsity patterns in the Hessian of the quadratic surface. We conduct numerical experiments on both artificial and public benchmark data sets to demonstrate the feasibility and effectiveness of the proposed models.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In binary classification, kernel-free linear or quadratic support vector machines are proposed to avoid dealing with difficulties such as finding appropriate kernel functions or tuning their hyper-parameters. Furthermore, Universum data points, which do not belong to any class, can be exploited to embed prior knowledge into the corresponding models so that the generalization performance is improved. In this paper, we design novel kernel-free Universum quadratic surface support vector machine models. Further, we propose the L1 norm regularized version that is beneficial for detecting potential sparsity patterns in the Hessian of the quadratic surface and reducing to the standard linear models if the data points are (almost) linearly separable. The proposed models are convex such that standard numerical solvers can be utilized for solving them. Nonetheless, we formulate a least squares version of the L1 norm regularized model and next, design an effective tailored algorithm that only requires solving one linear system. Several theoretical properties of these models are then reported/proved as well. We finally conduct numerical experiments on both artificial and public benchmark data sets to demonstrate the feasibility and effectiveness of the proposed models.

Related papers

• Robust kernel-free quadratic surface twin support vector machine with capped $L_1$-norm distance metric [0.46040036610482665]
  This paper proposes a robust capped L_norm kernel-free surface twin support vector machine (CL_QTSVM) The robustness of our model is further improved by employing the capped L_norm distance metric. An iterative algorithm is developed to efficiently solve the proposed model.
  arXiv  Detail & Related papers  (2024-05-27T09:23:52Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models. We provide an introduction and review of recent results on these topics, aimed at readers with backgrounds in physics and deep learning.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• Representer Point Selection for Explaining Regularized High-dimensional Models [105.75758452952357]
  We introduce a class of sample-based explanations we term high-dimensional representers. Our workhorse is a novel representer theorem for general regularized high-dimensional models. We study the empirical performance of our proposed methods on three real-world binary classification datasets and two recommender system datasets.
  arXiv  Detail & Related papers  (2023-05-31T16:23:58Z)
• Symplectic model reduction of Hamiltonian systems using data-driven quadratic manifolds [0.559239450391449]
  We present two novel approaches for the symplectic model reduction of high-dimensional Hamiltonian systems. The addition of quadratic terms to the state approximation, which sits at the heart of the proposed methodologies, enables us to better represent intrinsic low-dimensionality.
  arXiv  Detail & Related papers  (2023-05-24T18:23:25Z)
• Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
  This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution. We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
  arXiv  Detail & Related papers  (2022-10-21T13:19:45Z)
• Low-Rank Constraints for Fast Inference in Structured Models [110.38427965904266]
  This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models. Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces.
  arXiv  Detail & Related papers  (2022-01-08T00:47:50Z)
• Spike-and-Slab Generalized Additive Models and Scalable Algorithms for High-Dimensional Data [0.0]
  We propose hierarchical generalized additive models (GAMs) to accommodate high-dimensional data. We consider the smoothing penalty for proper shrinkage of curve and separation of smoothing function linear and nonlinear spaces. Two and deterministic algorithms, EM-Coordinate Descent and EM-Iterative Weighted Least Squares, are developed for different utilities.
  arXiv  Detail & Related papers  (2021-10-27T14:11:13Z)
• Hessian Eigenspectra of More Realistic Nonlinear Models [73.31363313577941]
  We make a emphprecise characterization of the Hessian eigenspectra for a broad family of nonlinear models. Our analysis takes a step forward to identify the origin of many striking features observed in more complex machine learning models.
  arXiv  Detail & Related papers  (2021-03-02T06:59:52Z)
• Non-parametric Models for Non-negative Functions [48.7576911714538]
  We provide the first model for non-negative functions from the same good linear models. We prove that it admits a representer theorem and provide an efficient dual formulation for convex problems.
  arXiv  Detail & Related papers  (2020-07-08T07:17:28Z)
• To Each Optimizer a Norm, To Each Norm its Generalization [31.682969645989512]
  We study the implicit regularization of optimization methods for linear models interpolating the training data in the under-parametrized and over-parametrized regimes. We argue that analyzing convergence to the standard maximum l2-margin is arbitrary and show that minimizing the norm induced by the data results in better generalizations.
  arXiv  Detail & Related papers  (2020-06-11T21:07:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.