Deep Ordinal Regression with Label Diversity

• URL: http://arxiv.org/abs/2006.15864v1
• Date: Mon, 29 Jun 2020 08:23:43 GMT
• Title: Deep Ordinal Regression with Label Diversity
• Authors: Axel Berg, Magnus Oskarsson and Mark O'Connor
• Abstract summary: We propose that using several discrete data representations simultaneously can improve neural network learning. Our approach is end-to-end differentiable and can be added as a simple extension to conventional learning methods.
• Score: 19.89482062012177
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Regression via classification (RvC) is a common method used for regression problems in deep learning, where the target variable belongs to a set of continuous values. By discretizing the target into a set of non-overlapping classes, it has been shown that training a classifier can improve neural network accuracy compared to using a standard regression approach. However, it is not clear how the set of discrete classes should be chosen and how it affects the overall solution. In this work, we propose that using several discrete data representations simultaneously can improve neural network learning compared to a single representation. Our approach is end-to-end differentiable and can be added as a simple extension to conventional learning methods, such as deep neural networks. We test our method on three challenging tasks and show that our method reduces the prediction error compared to a baseline RvC approach while maintaining a similar model complexity.

Related papers

• CBR - Boosting Adaptive Classification By Retrieval of Encrypted Network Traffic with Out-of-distribution [9.693391036125908]
  One of the common approaches is using Machine learning or Deep Learning-based solutions on a fixed number of classes. One of the solutions for handling unknown classes is to retrain the model, however, retraining models every time they become obsolete is both resource and time-consuming. In this paper, we introduce Adaptive Classification By Retrieval CBR, a novel approach for encrypted network traffic classification.
  arXiv  Detail & Related papers  (2024-03-17T13:14:09Z)
• Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach [102.0769560460338]
  We develop a simple logits approach (LORT) without the requirement of prior knowledge of the number of samples per class. Our method achieves state-of-the-art performance on various imbalanced datasets, including CIFAR100-LT, ImageNet-LT, and iNaturalist 2018.
  arXiv  Detail & Related papers  (2024-03-01T03:27:08Z)
• Continual Learning via Sequential Function-Space Variational Inference [65.96686740015902]
  We propose an objective derived by formulating continual learning as sequential function-space variational inference. Compared to objectives that directly regularize neural network predictions, the proposed objective allows for more flexible variational distributions. We demonstrate that, across a range of task sequences, neural networks trained via sequential function-space variational inference achieve better predictive accuracy than networks trained with related methods.
  arXiv  Detail & Related papers  (2023-12-28T18:44:32Z)
• Towards Diverse Evaluation of Class Incremental Learning: A Representation Learning Perspective [67.45111837188685]
  Class incremental learning (CIL) algorithms aim to continually learn new object classes from incrementally arriving data. We experimentally analyze neural network models trained by CIL algorithms using various evaluation protocols in representation learning.
  arXiv  Detail & Related papers  (2022-06-16T11:44:11Z)
• Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization [118.50301177912381]
  We show that Adam can converge to different solutions of the objective with provably different errors, even with weight decay globalization. We show that if convex, and the weight decay regularization is employed, any optimization algorithms including Adam will converge to the same solution.
  arXiv  Detail & Related papers  (2021-08-25T17:58:21Z)
• Class-Incremental Learning with Generative Classifiers [6.570917734205559]
  We propose a new strategy for class-incremental learning: generative classification. Our proposal is to learn the joint distribution p(x,y), factorized as p(x|y)p(y), and to perform classification using Bayes' rule. As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned.
  arXiv  Detail & Related papers  (2021-04-20T16:26:14Z)
• Meta-learning representations for clustering with infinite Gaussian mixture models [39.56814839510978]
  We propose a meta-learning method that train neural networks for obtaining representations such that clustering performance improves. The proposed method can cluster unseen unlabeled data using knowledge meta-learned with labeled data that are different from the unlabeled data.
  arXiv  Detail & Related papers  (2021-03-01T02:05:31Z)
• Learning a binary search with a recurrent neural network. A novel approach to ordinal regression analysis [0.0]
  This article investigates the application of sequence-to-sequence learning methods provided by the deep learning framework in ordinal regression. A method for visualizing the model's explanatory variables according to the ordinal target variable is proposed. The method is compared to traditional ordinal regression methods on a number of benchmark dataset, and is shown to have comparable or significantly better predictive power.
  arXiv  Detail & Related papers  (2021-01-07T16:16:43Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)
• Regularizing Class-wise Predictions via Self-knowledge Distillation [80.76254453115766]
  We propose a new regularization method that penalizes the predictive distribution between similar samples. This results in regularizing the dark knowledge (i.e., the knowledge on wrong predictions) of a single network. Our experimental results on various image classification tasks demonstrate that the simple yet powerful method can significantly improve the generalization ability.
  arXiv  Detail & Related papers  (2020-03-31T06:03:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.