Loss Functions and Metrics in Deep Learning
- URL: http://arxiv.org/abs/2307.02694v2
- Date: Wed, 6 Sep 2023 16:53:24 GMT
- Title: Loss Functions and Metrics in Deep Learning
- Authors: Juan Terven, Diana M. Cordova-Esparza, Alfonso Ramirez-Pedraza, Edgar
A. Chavez-Urbiola
- Abstract summary: This paper reviews the most prevalent loss functions and performance measurements in deep learning.
We examine the benefits and limits of each technique and illustrate their application to various deep-learning problems.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: One of the essential components of deep learning is the choice of the loss
function and performance metrics used to train and evaluate models. This paper
reviews the most prevalent loss functions and performance measurements in deep
learning. We examine the benefits and limits of each technique and illustrate
their application to various deep-learning problems. Our review aims to give a
comprehensive picture of the different loss functions and performance
indicators used in the most common deep learning tasks and help practitioners
choose the best method for their specific task.
Related papers
- Meta-Learning Loss Functions for Deep Neural Networks [2.4258031099152735]
This thesis explores the concept of meta-learning to improve performance, through the often-overlooked component of the loss function.
The loss function is a vital component of a learning system, as it represents the primary learning objective, where success is determined and quantified by the system's ability to optimize for that objective successfully.
arXiv Detail & Related papers (2024-06-14T04:46:14Z) - Fast and Efficient Local Search for Genetic Programming Based Loss
Function Learning [12.581217671500887]
We propose a new meta-learning framework for task and model-agnostic loss function learning via a hybrid search approach.
Results show that the learned loss functions bring improved convergence, sample efficiency, and inference performance on tabulated, computer vision, and natural language processing problems.
arXiv Detail & Related papers (2024-03-01T02:20:04Z) - Generalization Performance of Transfer Learning: Overparameterized and
Underparameterized Regimes [61.22448274621503]
In real-world applications, tasks often exhibit partial similarity, where certain aspects are similar while others are different or irrelevant.
Our study explores various types of transfer learning, encompassing two options for parameter transfer.
We provide practical guidelines for determining the number of features in the common and task-specific parts for improved generalization performance.
arXiv Detail & Related papers (2023-06-08T03:08:40Z) - Online Loss Function Learning [13.744076477599707]
Loss function learning aims to automate the task of designing a loss function for a machine learning model.
We propose a new loss function learning technique for adaptively updating the loss function online after each update to the base model parameters.
arXiv Detail & Related papers (2023-01-30T19:22:46Z) - A survey and taxonomy of loss functions in machine learning [60.41650195728953]
Most state-of-the-art machine learning techniques revolve around the optimisation of loss functions.
This survey aims to provide a reference of the most essential loss functions for both beginner and advanced machine learning practitioners.
arXiv Detail & Related papers (2023-01-13T14:38:24Z) - Learning Symbolic Model-Agnostic Loss Functions via Meta-Learning [12.581217671500887]
We propose a new meta-learning framework for learning model-agnostic loss functions via a hybrid neuro-symbolic search approach.
Results show that the meta-learned loss functions discovered by the newly proposed method outperform both the cross-entropy loss and state-of-the-art loss function learning methods.
arXiv Detail & Related papers (2022-09-19T10:29:01Z) - Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning [50.59295648948287]
In few-shot learning scenarios, the challenge is to generalize and perform well on new unseen examples.
We introduce a new meta-learning framework with a loss function that adapts to each task.
Our proposed framework, named Meta-Learning with Task-Adaptive Loss Function (MeTAL), demonstrates the effectiveness and the flexibility across various domains.
arXiv Detail & Related papers (2021-10-08T06:07:21Z) - AutoLoss-Zero: Searching Loss Functions from Scratch for Generic Tasks [78.27036391638802]
AutoLoss-Zero is the first framework for searching loss functions from scratch for generic tasks.
A loss-rejection protocol and a gradient-equivalence-check strategy are developed so as to improve the search efficiency.
Experiments on various computer vision tasks demonstrate that our searched loss functions are on par with or superior to existing loss functions.
arXiv Detail & Related papers (2021-03-25T17:59:09Z) - Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation [56.343646789922545]
We propose to automate the design of metric-specific loss functions by searching differentiable surrogate losses for each metric.
Experiments on PASCAL VOC and Cityscapes demonstrate that the searched surrogate losses outperform the manually designed loss functions consistently.
arXiv Detail & Related papers (2020-10-15T17:59:08Z) - Influence Functions in Deep Learning Are Fragile [52.31375893260445]
influence functions approximate the effect of samples in test-time predictions.
influence estimates are fairly accurate for shallow networks.
Hessian regularization is important to get highquality influence estimates.
arXiv Detail & Related papers (2020-06-25T18:25:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.