Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration

• URL: http://arxiv.org/abs/2407.06844v1
• Date: Tue, 9 Jul 2024 13:26:21 GMT
• Title: Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibration
• Authors: Tianshui Chen, Weihang Wang, Tao Pu, Jinghui Qin, Zhijing Yang, Jie Liu, Liang Lin,
• Abstract summary: This paper introduces the Multi-Label Confidence task, aiming to provide well-calibrated confidence scores in multi-label scenarios. Existing single-label calibration methods fail to account for category correlations, which are crucial for addressing semantic confusion. We propose the Dynamic Correlation Learning and Regularization algorithm, which leverages multi-grained semantic correlations to better model semantic confusion.
• Score: 60.95748658638956
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern visual recognition models often display overconfidence due to their reliance on complex deep neural networks and one-hot target supervision, resulting in unreliable confidence scores that necessitate calibration. While current confidence calibration techniques primarily address single-label scenarios, there is a lack of focus on more practical and generalizable multi-label contexts. This paper introduces the Multi-Label Confidence Calibration (MLCC) task, aiming to provide well-calibrated confidence scores in multi-label scenarios. Unlike single-label images, multi-label images contain multiple objects, leading to semantic confusion and further unreliability in confidence scores. Existing single-label calibration methods, based on label smoothing, fail to account for category correlations, which are crucial for addressing semantic confusion, thereby yielding sub-optimal performance. To overcome these limitations, we propose the Dynamic Correlation Learning and Regularization (DCLR) algorithm, which leverages multi-grained semantic correlations to better model semantic confusion for adaptive regularization. DCLR learns dynamic instance-level and prototype-level similarities specific to each category, using these to measure semantic correlations across different categories. With this understanding, we construct adaptive label vectors that assign higher values to categories with strong correlations, thereby facilitating more effective regularization. We establish an evaluation benchmark, re-implementing several advanced confidence calibration algorithms and applying them to leading multi-label recognition (MLR) models for fair comparison. Through extensive experiments, we demonstrate the superior performance of DCLR over existing methods in providing reliable confidence scores in multi-label scenarios.

Related papers

• Dual-Decoupling Learning and Metric-Adaptive Thresholding for Semi-Supervised Multi-Label Learning [81.83013974171364]
  Semi-supervised multi-label learning (SSMLL) is a powerful framework for leveraging unlabeled data to reduce the expensive cost of collecting precise multi-label annotations. Unlike semi-supervised learning, one cannot select the most probable label as the pseudo-label in SSMLL due to multiple semantics contained in an instance. We propose a dual-perspective method to generate high-quality pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-26T09:33:53Z)
• Confidence Self-Calibration for Multi-Label Class-Incremental Learning [21.104984143597882]
  Partial label challenge in Multi-Label Class-Incremental Learning (MLCIL) arises when only the new classes are labeled during training. This issue leads to a proliferation of false-positive errors due to erroneously high confidence multi-label predictions. We propose a Confidence Self-Calibration (CSC) approach to refine multi-label confidence calibration in MLCIL.
  arXiv  Detail & Related papers  (2024-03-19T09:14:52Z)
• Binary Classification with Confidence Difference [100.08818204756093]
  This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
  arXiv  Detail & Related papers  (2023-10-09T11:44:50Z)
• Perception and Semantic Aware Regularization for Sequential Confidence Calibration [12.265757315192497]
  We propose a Perception and Semantic aware Sequence Regularization framework. We introduce a semantic context-free recognition and a language model to acquire similar sequences with high perceptive similarities and semantic correlation. Experiments on canonical sequence recognition tasks, including scene text and speech recognition, demonstrate that our method sets novel state-of-the-art results.
  arXiv  Detail & Related papers  (2023-05-31T02:16:29Z)
• A Confidence-based Partial Label Learning Model for Crowd-Annotated Named Entity Recognition [74.79785063365289]
  Existing models for named entity recognition (NER) are mainly based on large-scale labeled datasets. We propose a Confidence-based Partial Label Learning (CPLL) method to integrate the prior confidence (given by annotators) and posterior confidences (learned by models) for crowd-annotated NER.
  arXiv  Detail & Related papers  (2023-05-21T15:31:23Z)
• Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning [97.88458953075205]
  Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
  arXiv  Detail & Related papers  (2023-05-04T12:52:18Z)
• Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning [146.11600461034746]
  Method for unsupervised meta-learning, CACTUs, is a clustering-based approach with pseudo-labeling. This approach is model-agnostic and can be combined with supervised algorithms to learn from unlabeled data. We prove that the core reason for this is lack of a clustering-friendly property in the embedding space.
  arXiv  Detail & Related papers  (2022-09-27T19:04:36Z)
• Evolving Multi-Label Fuzzy Classifier [5.53329677986653]
  Multi-label classification has attracted much attention in the machine learning community to address the problem of assigning single samples to more than one class at the same time. We propose an evolving multi-label fuzzy classifier (EFC-ML) which is able to self-adapt and self-evolve its structure with new incoming multi-label samples in an incremental, single-pass manner.
  arXiv  Detail & Related papers  (2022-03-29T08:01:03Z)
• Label fusion and training methods for reliable representation of inter-rater uncertainty [0.0]
  Training deep learning networks with annotations from multiple raters mitigates the model's bias towards a single expert. Various methods exist to take into account different expert labels. We compare three label fusion methods: STAPLE, average of the rater's segmentation, and random sampling each rater's segmentation during training.
  arXiv  Detail & Related papers  (2022-02-15T16:35:47Z)
• Evolving Multi-label Classification Rules by Exploiting High-order Label Correlation [2.9822184411723645]
  In multi-label classification tasks, each problem instance is associated with multiple classes simultaneously. The correlation between labels can be exploited at different levels such as capturing the pair-wise correlation or exploiting the higher-order correlations. This paper aims at exploiting the high-order label correlation within subsets of labels using a supervised learning classifier system.
  arXiv  Detail & Related papers  (2020-07-22T18:13:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.