AML-SVM: Adaptive Multilevel Learning with Support Vector Machines

• URL: http://arxiv.org/abs/2011.02592v1
• Date: Thu, 5 Nov 2020 00:17:02 GMT
• Title: AML-SVM: Adaptive Multilevel Learning with Support Vector Machines
• Authors: Ehsan Sadrfaridpour, Korey Palmer, Ilya Safro (Clemson University)
• Abstract summary: This paper proposes an adaptive multilevel learning framework for the nonlinear SVM. It improves the classification quality across the refinement process, and leverages multi-threaded parallel processing for better performance.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The support vector machines (SVM) is one of the most widely used and practical optimization based classification models in machine learning because of its interpretability and flexibility to produce high quality results. However, the big data imposes a certain difficulty to the most sophisticated but relatively slow versions of SVM, namely, the nonlinear SVM. The complexity of nonlinear SVM solvers and the number of elements in the kernel matrix quadratically increases with the number of samples in training data. Therefore, both runtime and memory requirements are negatively affected. Moreover, the parameter fitting has extra kernel parameters to tune, which exacerbate the runtime even further. This paper proposes an adaptive multilevel learning framework for the nonlinear SVM, which addresses these challenges, improves the classification quality across the refinement process, and leverages multi-threaded parallel processing for better performance. The integration of parameter fitting in the hierarchical learning framework and adaptive process to stop unnecessary computation significantly reduce the running time while increase the overall performance. The experimental results demonstrate reduced variance on prediction over validation and test data across levels in the hierarchy, and significant speedup compared to state-of-the-art nonlinear SVM libraries without a decrease in the classification quality. The code is accessible at https://github.com/esadr/amlsvm.

Related papers

• Sparse Learning and Class Probability Estimation with Weighted Support Vector Machines [1.3597551064547502]
  weighted Support Vector Machines (wSVMs) have shown great values in robustly predicting the class probability and classification for various problems with high accuracy. We propose novel wSVMs frameworks that incorporate automatic variable selection with accurate probability estimation for sparse learning problems. The proposed wSVMs-based sparse learning methods have wide applications and can be further extended to $K$-class problems through ensemble learning.
  arXiv  Detail & Related papers  (2023-12-17T06:12:33Z)
• Optimization meets Machine Learning: An Exact Algorithm for Semi-Supervised Support Vector Machines [0.9831489366502302]
  Support vector machines (SVMs) are well-studied supervised learning models for binary classification. We present a new branch approach for S3VMs using semidefinite programming (SDP) relaxations. SDP relaxation provides bounds significantly stronger than the ones available in the literature.
  arXiv  Detail & Related papers  (2023-12-15T13:44:54Z)
• AdaLomo: Low-memory Optimization with Adaptive Learning Rate [59.64965955386855]
  We introduce low-memory optimization with adaptive learning rate (AdaLomo) for large language models. AdaLomo results on par with AdamW, while significantly reducing memory requirements, thereby lowering the hardware barrier to training large language models.
  arXiv  Detail & Related papers  (2023-10-16T09:04:28Z)
• Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimization [73.17488635491262]
  Support vector machines (SVMs) are popular learning algorithms to deal with binary classification problems. This article introduces EBCS-SVM: evolutionary bilevel cost-sensitive SVMs.
  arXiv  Detail & Related papers  (2022-04-21T16:08:44Z)
• Max-Margin Contrastive Learning [120.32963353348674]
  We present max-margin contrastive learning (MMCL) for unsupervised representation learning. Our approach selects negatives as the sparse support vectors obtained via a quadratic optimization problem. We validate our approach on standard vision benchmark datasets, demonstrating better performance in unsupervised representation learning.
  arXiv  Detail & Related papers  (2021-12-21T18:56:54Z)
• Training very large scale nonlinear SVMs using Alternating Direction Method of Multipliers coupled with the Hierarchically Semi-Separable kernel approximations [0.0]
  nonlinear Support Vector Machines (SVMs) produce significantly higher classification quality when compared to linear ones. Their computational complexity is prohibitive for large-scale datasets.
  arXiv  Detail & Related papers  (2021-08-09T16:52:04Z)
• Estimating Average Treatment Effects with Support Vector Machines [77.34726150561087]
  Support vector machine (SVM) is one of the most popular classification algorithms in the machine learning literature. We adapt SVM as a kernel-based weighting procedure that minimizes the maximum mean discrepancy between the treatment and control groups. We characterize the bias of causal effect estimation arising from this trade-off, connecting the proposed SVM procedure to the existing kernel balancing methods.
  arXiv  Detail & Related papers  (2021-02-23T20:22:56Z)
• Unsupervised Real Time Prediction of Faults Using the Support Vector Machine [1.1852751647387592]
  We show that the proposed solution can perform much better when using the SMO training algorithm. The classification performance of this predictive model is considerably better than the SVM with and without SMO training algorithm.
  arXiv  Detail & Related papers  (2020-12-30T04:27:10Z)
• A fast learning algorithm for One-Class Slab Support Vector Machines [1.1613446814180841]
  This paper proposes fast training method for One Class Slab SVMs using an updated Sequential Minimal Optimization (SMO) The results indicate that this training method scales better to large sets of training data than other Quadratic Programming (QP) solvers.
  arXiv  Detail & Related papers  (2020-11-06T09:16:39Z)
• Scaling Distributed Deep Learning Workloads beyond the Memory Capacity with KARMA [58.040931661693925]
  We propose a strategy that combines redundant recomputing and out-of-core methods. We achieve an average of 1.52x speedup in six different models over the state-of-the-art out-of-core methods. Our data parallel out-of-core solution can outperform complex hybrid model parallelism in training large models, e.g. Megatron-LM and Turning-NLG.
  arXiv  Detail & Related papers  (2020-08-26T07:24:34Z)
• On Coresets for Support Vector Machines [61.928187390362176]
  A coreset is a small, representative subset of the original data points. We show that our algorithm can be used to extend the applicability of any off-the-shelf SVM solver to streaming, distributed, and dynamic data settings.
  arXiv  Detail & Related papers  (2020-02-15T23:25:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.