A Survey of Learning Criteria Going Beyond the Usual Risk

• URL: http://arxiv.org/abs/2110.04996v3
• Date: Thu, 30 Nov 2023 00:09:25 GMT
• Title: A Survey of Learning Criteria Going Beyond the Usual Risk
• Authors: Matthew J. Holland and Kazuki Tanabe
• Abstract summary: "Good performance" is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data. While optimizing for performance on average is intuitive, convenient to analyze in theory, and easy to implement in practice, such a choice brings about trade-offs.
• Score: 7.335712499936906
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Virtually all machine learning tasks are characterized using some form of loss function, and "good performance" is typically stated in terms of a sufficiently small average loss, taken over the random draw of test data. While optimizing for performance on average is intuitive, convenient to analyze in theory, and easy to implement in practice, such a choice brings about trade-offs. In this work, we survey and introduce a wide variety of non-traditional criteria used to design and evaluate machine learning algorithms, place the classical paradigm within the proper historical context, and propose a view of learning problems which emphasizes the question of "what makes for a desirable loss distribution?" in place of tacit use of the expected loss.

Related papers

• Embedding generalization within the learning dynamics: An approach based-on sample path large deviation theory [0.0]
  We consider an empirical risk perturbation based learning problem that exploits methods from continuous-time perspective. We provide an estimate in the small noise limit based on the Freidlin-Wentzell theory of large deviations. We also present a computational algorithm that solves the corresponding variational problem leading to an optimal point estimates.
  arXiv  Detail & Related papers  (2024-08-04T23:31:35Z)
• Asymptotically Optimal Regret for Black-Box Predict-then-Optimize [7.412445894287709]
  We study new black-box predict-then-optimize problems that lack special structure and where we only observe the reward from the action taken. We present a novel loss function, which we call Empirical Soft Regret (ESR), designed to significantly improve reward when used in training. We also show our approach significantly outperforms state-of-the-art algorithms on real-world decision-making problems in news recommendation and personalized healthcare.
  arXiv  Detail & Related papers  (2024-06-12T04:46:23Z)
• Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize [57.22851616806617]
  We show that our method achieves state-of-the-art results in four domains from the literature. Our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.
  arXiv  Detail & Related papers  (2023-05-26T11:17:45Z)
• A survey and taxonomy of loss functions in machine learning [60.41650195728953]
  Most state-of-the-art machine learning techniques revolve around the optimisation of loss functions. This survey aims to provide a reference of the most essential loss functions for both beginner and advanced machine learning practitioners.
  arXiv  Detail & Related papers  (2023-01-13T14:38:24Z)
• A Survey on Deep Industrial Transfer Learning in Fault Prognostics [0.0]
  This paper aims at establishing best practices for future research in this field. It is shown that the field is lacking common benchmarks to robustly compare results and facilitate scientific progress. The data sets utilized in these publications are surveyed as well in order to identify suitable candidates for such benchmark scenarios.
  arXiv  Detail & Related papers  (2023-01-04T17:01:27Z)
• Prototype-Anchored Learning for Learning with Imperfect Annotations [83.7763875464011]
  It is challenging to learn unbiased classification models from imperfectly annotated datasets. We propose a prototype-anchored learning (PAL) method, which can be easily incorporated into various learning-based classification schemes. We verify the effectiveness of PAL on class-imbalanced learning and noise-tolerant learning by extensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-06-23T10:25:37Z)
• Learning Towards the Largest Margins [83.7763875464011]
  Loss function should promote the largest possible margins for both classes and samples. Not only does this principled framework offer new perspectives to understand and interpret existing margin-based losses, but it can guide the design of new tools.
  arXiv  Detail & Related papers  (2022-06-23T10:03:03Z)
• CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learning [55.733193075728096]
  Modern deep neural networks can easily overfit to biased training data containing corrupted labels or class imbalance. Sample re-weighting methods are popularly used to alleviate this data bias issue. We propose a meta-model capable of adaptively learning an explicit weighting scheme directly from data.
  arXiv  Detail & Related papers  (2022-02-11T13:49:51Z)
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes [85.5691102676175]
  We propose a simple but effective debiasing technique in an unsupervised manner. We perform clustering on the feature embedding space and identify pseudoattributes by taking advantage of the clustering results. We then employ a novel cluster-based reweighting scheme for learning debiased representation.
  arXiv  Detail & Related papers  (2021-08-06T05:20:46Z)
• Accuracy and Fairness Trade-offs in Machine Learning: A Stochastic Multi-Objective Approach [0.0]
  In the application of machine learning to real-life decision-making systems, the prediction outcomes might discriminate against people with sensitive attributes, leading to unfairness. The commonly used strategy in fair machine learning is to include fairness as a constraint or a penalization term in the minimization of the prediction loss. In this paper, we introduce a new approach to handle fairness by formulating a multi-objective optimization problem.
  arXiv  Detail & Related papers  (2020-08-03T18:51:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.