Solving for multi-class using orthogonal coding matrices

• URL: http://arxiv.org/abs/1801.09055v6
• Date: Wed, 17 May 2023 15:54:55 GMT
• Title: Solving for multi-class using orthogonal coding matrices
• Authors: Peter Mills
• Abstract summary: Error correcting code (ECC) is a common method of generalizing binary to multi-class classification. Here we test two types of orthogonal ECCs on seven different datasets. We compare them with three other multi-class methods: 1 vs. 1, one-versus-the-rest and random ECCs.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A common method of generalizing binary to multi-class classification is the error correcting code (ECC). ECCs may be optimized in a number of ways, for instance by making them orthogonal. Here we test two types of orthogonal ECCs on seven different datasets using three types of binary classifier and compare them with three other multi-class methods: 1 vs. 1, one-versus-the-rest and random ECCs. The first type of orthogonal ECC, in which the codes contain no zeros, admits a fast and simple method of solving for the probabilities. Orthogonal ECCs are always more accurate than random ECCs as predicted by recent literature. Improvments in uncertainty coefficient (U.C.) range between 0.4--17.5% (0.004--0.139, absolute), while improvements in Brier score between 0.7--10.7%. Unfortunately, orthogonal ECCs are rarely more accurate than 1 vs. 1. Disparities are worst when the methods are paired with logistic regression, with orthogonal ECCs never beating 1 vs. 1. When the methods are paired with SVM, the losses are less significant, peaking at 1.5%, relative, 0.011 absolute in uncertainty coefficient and 6.5% in Brier scores. Orthogonal ECCs are always the fastest of the five multi-class methods when paired with linear classifiers. When paired with a piecewise linear classifier, whose classification speed does not depend on the number of training samples, classifications using orthogonal ECCs were always more accurate than the other methods and also faster than 1 vs. 1. Losses against 1 vs. 1 here were higher, peaking at 1.9% (0.017, absolute), in U.C. and 39% in Brier score. Gains in speed ranged between 1.1% and over 100%. Whether the speed increase is worth the penalty in accuracy will depend on the application.

Related papers

• Risk-based Calibration for Generative Classifiers [4.792851066169872]
  We propose a learning procedure called risk-based calibration (RC) RC iteratively refines the generative classifier by adjusting its joint probability distribution according to the 0-1 loss in training samples. RC significantly outperforms closed-form learning procedures in terms of both training error and generalization error.
  arXiv  Detail & Related papers  (2024-09-05T14:06:56Z)
• MOKD: Cross-domain Finetuning for Few-shot Classification via Maximizing Optimized Kernel Dependence [97.93517982908007]
  In cross-domain few-shot classification, NCC aims to learn representations to construct a metric space where few-shot classification can be performed. In this paper, we find that there exist high similarities between NCC-learned representations of two samples from different classes. We propose a bi-level optimization framework, emphmaximizing optimized kernel dependence (MOKD) to learn a set of class-specific representations that match the cluster structures indicated by labeled data.
  arXiv  Detail & Related papers  (2024-05-29T05:59:52Z)
• Block-regularized 5$\times$2 Cross-validated McNemar's Test for Comparing Two Classification Algorithms [5.7490445900906835]
  Cross-validation method repeats the HO method in multiple times and produces a stable estimation. A block-regularized 5$times$2 CV (BCV) has been shown in many previous studies to be superior to the other CV methods. We demonstrate the reasonable type I error and the promising power of the proposed 5$times$2 BCV McNemar's test on simulated and real-world data sets.
  arXiv  Detail & Related papers  (2023-04-08T11:35:19Z)
• Orthogonal SVD Covariance Conditioning and Latent Disentanglement [65.67315418971688]
  Inserting an SVD meta-layer into neural networks is prone to make the covariance ill-conditioned. We propose Nearest Orthogonal Gradient (NOG) and Optimal Learning Rate (OLR) Experiments on visual recognition demonstrate that our methods can simultaneously improve covariance conditioning and generalization.
  arXiv  Detail & Related papers  (2022-12-11T20:31:31Z)
• Improved techniques for deterministic l2 robustness [63.34032156196848]
  Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_2$ norm is useful for adversarial robustness, interpretable gradients and stable training. We introduce a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer. We significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 and CIFAR-100.
  arXiv  Detail & Related papers  (2022-11-15T19:10:12Z)
• Random Convolution Kernels with Multi-Scale Decomposition for Preterm EEG Inter-burst Detection [0.0]
  Linear classifiers with random convolution kernels are computationally efficient methods that need no design or domain knowledge. A recently proposed method, RandOm Convolutional KErnel Transforms, has shown high accuracy across a range of time-series data sets. We propose a multi-scale version of this method, using both high- and low-frequency components.
  arXiv  Detail & Related papers  (2021-08-04T13:07:41Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)
• A Link between Coding Theory and Cross-Validation with Applications [2.287027904771153]
  We show that the exact answers are given by the theory of error detecting codes. As an immediate practical application, we develop new LPOCV based randomization tests for learning algorithms.
  arXiv  Detail & Related papers  (2021-03-22T13:57:45Z)
• FROCC: Fast Random projection-based One-Class Classification [4.312746668772342]
  Fast Random projection-based One-Class Classification (FROCC) is an efficient method for one-class classification. FROCC achieves up to 3.1 percent points better ROC, with 1.2--67.8x speedup in training and test times.
  arXiv  Detail & Related papers  (2020-11-29T08:56:59Z)
• Minimum Variance Embedded Auto-associative Kernel Extreme Learning Machine for One-class Classification [1.4146420810689422]
  VAAKELM is a novel extension of an auto-associative kernel extreme learning machine. It embeds minimum variance information within its architecture and reduces the intra-class variance. It follows a reconstruction-based approach to one-class classification and minimizes the reconstruction error.
  arXiv  Detail & Related papers  (2020-11-24T17:00:30Z)
• Re-Assessing the "Classify and Count" Quantification Method [88.60021378715636]
  "Classify and Count" (CC) is often a biased estimator. Previous works have failed to use properly optimised versions of CC. We argue that, while still inferior to some cutting-edge methods, they deliver near-state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2020-11-04T21:47:39Z)
• Self-Weighted Robust LDA for Multiclass Classification with Edge Classes [111.5515086563592]
  A novel self-weighted robust LDA with l21-norm based between-class distance criterion, called SWRLDA, is proposed for multi-class classification. The proposed SWRLDA is easy to implement, and converges fast in practice.
  arXiv  Detail & Related papers  (2020-09-24T12:32:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.