XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners

• URL: http://arxiv.org/abs/2310.05502v3
• Date: Fri, 15 Mar 2024 02:30:28 GMT
• Title: XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners
• Authors: Yun Luo, Zhen Yang, Fandong Meng, Yingjie Li, Fang Guo, Qinglin Qi, Jie Zhou, Yue Zhang,
• Abstract summary: We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
• Score: 71.8257151788923
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Active learning (AL), which aims to construct an effective training set by iteratively curating the most formative unlabeled data for annotation, has been widely used in low-resource tasks. Most active learning techniques in classification rely on the model's uncertainty or disagreement to choose unlabeled data, suffering from the problem of over-confidence in superficial patterns and a lack of exploration. Inspired by the cognitive processes in which humans deduce and predict through causal information, we take an initial attempt towards integrating rationales into AL and propose a novel Explainable Active Learning framework (XAL) for low-resource text classification, which aims to encourage classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Specifically, besides using a pre-trained bi-directional encoder for classification, we employ a pre-trained uni-directional decoder to generate and score the explanation. We further facilitate the alignment of the model with human reasoning preference through a proposed ranking loss. During the selection of unlabeled data, the predicted uncertainty of the encoder and the explanation score of the decoder complement each other as the final metric to acquire informative data. Extensive experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines. Analysis indicates that the proposed method can generate corresponding explanations for its predictions.

Related papers

• Semi-Supervised Variational Adversarial Active Learning via Learning to Rank and Agreement-Based Pseudo Labeling [6.771578432805963]
  Active learning aims to alleviate the amount of labor involved in data labeling by automating the selection of unlabeled samples. We introduce novel techniques that significantly improve the use of abundant unlabeled data during training. We demonstrate the superior performance of our approach over the state of the art on various image classification and segmentation benchmark datasets.
  arXiv  Detail & Related papers  (2024-08-23T00:35:07Z)
• Unsupervised Transfer Learning via Adversarial Contrastive Training [3.227277661633986]
  We propose a novel unsupervised transfer learning approach using adversarial contrastive training (ACT) Our experimental results demonstrate outstanding classification accuracy with both fine-tuned linear probe and K-NN protocol across various datasets.
  arXiv  Detail & Related papers  (2024-08-16T05:11:52Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning [69.81438976273866]
  Open-set semi-supervised learning (Open-set SSL) considers a more practical scenario, where unlabeled data and test data contain new categories (outliers) not observed in labeled data (inliers) We introduce evidential deep learning (EDL) as an outlier detector to quantify different types of uncertainty, and design different uncertainty metrics for self-training and inference. We propose a novel adaptive negative optimization strategy, making EDL more tailored to the unlabeled dataset containing both inliers and outliers.
  arXiv  Detail & Related papers  (2023-03-21T09:07:15Z)
• NorMatch: Matching Normalizing Flows with Discriminative Classifiers for Semi-Supervised Learning [8.749830466953584]
  Semi-Supervised Learning (SSL) aims to learn a model using a tiny labeled set and massive amounts of unlabeled data. In this work we introduce a new framework for SSL named NorMatch. We demonstrate, through numerical and visual results, that NorMatch achieves state-of-the-art performance on several datasets.
  arXiv  Detail & Related papers  (2022-11-17T15:39:18Z)
• Mutual Information Learned Classifiers: an Information-theoretic Viewpoint of Training Deep Learning Classification Systems [9.660129425150926]
  Cross entropy loss can easily lead us to find models which demonstrate severe overfitting behavior. In this paper, we prove that the existing cross entropy loss minimization for training DNN classifiers essentially learns the conditional entropy of the underlying data distribution. We propose a mutual information learning framework where we train DNN classifiers via learning the mutual information between the label and input.
  arXiv  Detail & Related papers  (2022-10-03T15:09:19Z)
• Low-Regret Active learning [64.36270166907788]
  We develop an online learning algorithm for identifying unlabeled data points that are most informative for training. At the core of our work is an efficient algorithm for sleeping experts that is tailored to achieve low regret on predictable (easy) instances.
  arXiv  Detail & Related papers  (2021-04-06T22:53:45Z)
• ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning [52.831894583501395]
  Continual learning assumes the incoming data are fully labeled, which might not be applicable in real applications. We propose deep Online Replay with Discriminator Consistency (ORDisCo) to interdependently learn a classifier with a conditional generative adversarial network (GAN) We show ORDisCo achieves significant performance improvement on various semi-supervised learning benchmark datasets for SSCL.
  arXiv  Detail & Related papers  (2021-01-02T09:04:14Z)
• Out-distribution aware Self-training in an Open World Setting [62.19882458285749]
  We leverage unlabeled data in an open world setting to further improve prediction performance. We introduce out-distribution aware self-training, which includes a careful sample selection strategy. Our classifiers are by design out-distribution aware and can thus distinguish task-related inputs from unrelated ones.
  arXiv  Detail & Related papers  (2020-12-21T12:25:04Z)
• Self-Supervised Relational Reasoning for Representation Learning [5.076419064097733]
  In self-supervised learning, a system is tasked with achieving a surrogate objective by defining alternative targets on unlabeled data. We propose a novel self-supervised formulation of relational reasoning that allows a learner to bootstrap a signal from information implicit in unlabeled data. We evaluate the proposed method following a rigorous experimental procedure, using standard datasets, protocols, and backbones.
  arXiv  Detail & Related papers  (2020-06-10T14:24:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.