Dirichlet-Based Coarse-to-Fine Example Selection For Open-Set Annotation

• URL: http://arxiv.org/abs/2409.17607v1
• Date: Thu, 26 Sep 2024 07:47:50 GMT
• Title: Dirichlet-Based Coarse-to-Fine Example Selection For Open-Set Annotation
• Authors: Ye-Wen Wang, Chen-Chen Zong, Ming-Kun Xie, Sheng-Jun Huang,
• Abstract summary: We propose a Dirichlet-based Coarse-to-Fine Example Selection (DCFS) strategy. Our method introduces simplex-based evidential deep learning (EDL) to break translation invariance. Experiments on various openness ratio datasets demonstrate that DCFS achieves state-of-art performance.
• Score: 37.33424244520009
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Active learning (AL) has achieved great success by selecting the most valuable examples from unlabeled data. However, they usually deteriorate in real scenarios where open-set noise gets involved, which is studied as open-set annotation (OSA). In this paper, we owe the deterioration to the unreliable predictions arising from softmax-based translation invariance and propose a Dirichlet-based Coarse-to-Fine Example Selection (DCFS) strategy accordingly. Our method introduces simplex-based evidential deep learning (EDL) to break translation invariance and distinguish known and unknown classes by considering evidence-based data and distribution uncertainty simultaneously. Furthermore, hard known-class examples are identified by model discrepancy generated from two classifier heads, where we amplify and alleviate the model discrepancy respectively for unknown and known classes. Finally, we combine the discrepancy with uncertainties to form a two-stage strategy, selecting the most informative examples from known classes. Extensive experiments on various openness ratio datasets demonstrate that DCFS achieves state-of-art performance.

Related papers

• Bidirectional Uncertainty-Based Active Learning for Open Set Annotation [28.624321743629825]
  Active learning in open set scenarios presents a novel challenge of identifying the most valuable examples. Traditional methods prioritize selecting informative examples with low confidence. Recent methods favor the most probable known-class examples. We propose a Bidirectional Uncertainty-based Active Learning framework.
  arXiv  Detail & Related papers  (2024-02-23T08:59:04Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Late Stopping: Avoiding Confidently Learning from Mislabeled Examples [61.00103151680946]
  We propose a new framework, Late Stopping, which leverages the intrinsic robust learning ability of DNNs through a prolonged training process. We empirically observe that mislabeled and clean examples exhibit differences in the number of epochs required for them to be consistently and correctly classified. Experimental results on benchmark-simulated and real-world noisy datasets demonstrate that the proposed method outperforms state-of-the-art counterparts.
  arXiv  Detail & Related papers  (2023-08-26T12:43:25Z)
• Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning [69.81438976273866]
  Open-set semi-supervised learning (Open-set SSL) considers a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers) We introduce evidential deep learning (EDL) as an outlier detector to quantify different types of uncertainty, and design different uncertainty metrics for self-training and inference. We propose a novel adaptive negative optimization strategy, making EDL more tailored to the unlabeled dataset containing both inliers and outliers.
  arXiv  Detail & Related papers  (2023-03-21T09:07:15Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Gray Learning from Non-IID Data with Out-of-distribution Samples [45.788789553551176]
  The integrity of training data, even when annotated by experts, is far from guaranteed. We introduce a novel approach, termed textitGray Learning, which leverages both ground-truth and complementary labels. By grounding our approach in statistical learning theory, we derive bounds for the generalization error, demonstrating that GL achieves tight constraints even in non-IID settings.
  arXiv  Detail & Related papers  (2022-06-19T10:46:38Z)
• Active Learning for Open-set Annotation [38.739845944840454]
  We propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-01-18T06:11:51Z)
• Dash: Semi-Supervised Learning with Dynamic Thresholding [72.74339790209531]
  We propose a semi-supervised learning (SSL) approach that uses unlabeled examples to train models. Our proposed approach, Dash, enjoys its adaptivity in terms of unlabeled data selection.
  arXiv  Detail & Related papers  (2021-09-01T23:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.