Simple and Robust Loss Design for Multi-Label Learning with Missing Labels

• URL: http://arxiv.org/abs/2112.07368v1
• Date: Mon, 13 Dec 2021 11:39:19 GMT
• Title: Simple and Robust Loss Design for Multi-Label Learning with Missing Labels
• Authors: Youcai Zhang, Yuhao Cheng, Xinyu Huang, Fei Wen, Rui Feng, Yaqian Li and Yandong Guo
• Abstract summary: We propose two simple yet effective methods via robust loss design based on an observation a model can identify missing labels during training. The first is a novel robust loss for negatives, namely the Hill loss, which re-weights negatives in the shape of a hill to alleviate the effect of false negatives. The second is a self-paced loss correction (SPLC) method, which uses a loss derived from the maximum likelihood criterion under an approximate distribution of missing labels.
• Score: 14.7306301893944
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-label learning in the presence of missing labels (MLML) is a challenging problem. Existing methods mainly focus on the design of network structures or training schemes, which increase the complexity of implementation. This work seeks to fulfill the potential of loss function in MLML without increasing the procedure and complexity. Toward this end, we propose two simple yet effective methods via robust loss design based on an observation that a model can identify missing labels during training with a high precision. The first is a novel robust loss for negatives, namely the Hill loss, which re-weights negatives in the shape of a hill to alleviate the effect of false negatives. The second is a self-paced loss correction (SPLC) method, which uses a loss derived from the maximum likelihood criterion under an approximate distribution of missing labels. Comprehensive experiments on a vast range of multi-label image classification datasets demonstrate that our methods can remarkably boost the performance of MLML and achieve new state-of-the-art loss functions in MLML.

Related papers

• Boosting Single Positive Multi-label Classification with Generalized Robust Loss [12.942661993684881]
  Multi-label learning (MLL) requires comprehensive multi-semantic annotations that is hard to fully obtain. In this paper, we investigate Single Positive Multi-label Learning (SPML), where each image is associated with merely one positive label. Existing SPML methods only focus on designing losses using mechanisms such as hard pseudo-labeling and robust losses.
  arXiv  Detail & Related papers  (2024-05-06T14:13:38Z)
• Querying Easily Flip-flopped Samples for Deep Active Learning [63.62397322172216]
  Active learning is a machine learning paradigm that aims to improve the performance of a model by strategically selecting and querying unlabeled data. One effective selection strategy is to base it on the model's predictive uncertainty, which can be interpreted as a measure of how informative a sample is. This paper proposes the it least disagree metric (LDM) as the smallest probability of disagreement of the predicted label.
  arXiv  Detail & Related papers  (2024-01-18T08:12:23Z)
• Decoupled Prototype Learning for Reliable Test-Time Adaptation [50.779896759106784]
  Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference. One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels. This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise. We propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation.
  arXiv  Detail & Related papers  (2024-01-15T03:33:39Z)
• Noise-Robust Loss Functions: Enhancing Bounded Losses for Large-Scale Noisy Data Learning [0.0]
  Large annotated datasets inevitably contain noisy labels, which poses a major challenge for training deep neural networks as they easily memorize the labels. Noise-robust loss functions have emerged as a notable strategy to counteract this issue, but it remains challenging to create a robust loss function which is not susceptible to underfitting. We propose a novel method denoted as logit bias, which adds a real number $epsilon$ to the logit at the position of the correct class.
  arXiv  Detail & Related papers  (2023-06-08T18:38:55Z)
• Learning in Imperfect Environment: Multi-Label Classification with Long-Tailed Distribution and Partial Labels [53.68653940062605]
  We introduce a novel task, Partial labeling and Long-Tailed Multi-Label Classification (PLT-MLC) We find that most LT-MLC and PL-MLC approaches fail to solve the degradation-MLC. We propose an end-to-end learning framework: textbfCOrrection $rightarrow$ textbfModificattextbfIon $rightarrow$ balantextbfCe.
  arXiv  Detail & Related papers  (2023-04-20T20:05:08Z)
• L2B: Learning to Bootstrap Robust Models for Combating Label Noise [52.02335367411447]
  This paper introduces a simple and effective method, named Learning to Bootstrap (L2B) It enables models to bootstrap themselves using their own predictions without being adversely affected by erroneous pseudo-labels. It achieves this by dynamically adjusting the importance weight between real observed and generated labels, as well as between different samples through meta-learning.
  arXiv  Detail & Related papers  (2022-02-09T05:57:08Z)
• Adaptive neighborhood Metric learning [184.95321334661898]
  We propose a novel distance metric learning algorithm, named adaptive neighborhood metric learning (ANML) ANML can be used to learn both the linear and deep embeddings. The emphlog-exp mean function proposed in our method gives a new perspective to review the deep metric learning methods.
  arXiv  Detail & Related papers  (2022-01-20T17:26:37Z)
• Rethinking Deep Contrastive Learning with Embedding Memory [58.66613563148031]
  Pair-wise loss functions have been extensively studied and shown to continuously improve the performance of deep metric learning (DML) We provide a new methodology for systematically studying weighting strategies of various pair-wise loss functions, and rethink pair weighting with an embedding memory.
  arXiv  Detail & Related papers  (2021-03-25T17:39:34Z)
• Evolving parametrized Loss for Image Classification Learning on Small Datasets [1.4685355149711303]
  This paper proposes a meta-learning approach to evolving a parametrized loss function, which is called Meta-Loss Network (MLN) In our approach, the MLN is embedded in the framework of classification learning as a differentiable objective function. Experiment results demonstrate that the MLN effectively improved generalization compared to classical cross-entropy error and mean squared error.
  arXiv  Detail & Related papers  (2021-03-15T10:00:18Z)
• Bidirectional Loss Function for Label Enhancement and Distribution Learning [23.61708127340584]
  Two challenges exist in LDL: how to address the dimensional gap problem during the learning process and how to recover label distributions from logical labels. This study considers bidirectional projections function which can be applied in LE and LDL problems simultaneously. Experiments on several real-world datasets are carried out to demonstrate the superiority of the proposed method for both LE and LDL.
  arXiv  Detail & Related papers  (2020-07-07T03:02:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.