SoQal: Selective Oracle Questioning in Active Learning

• URL: http://arxiv.org/abs/2004.10468v1
• Date: Wed, 22 Apr 2020 09:53:55 GMT
• Title: SoQal: Selective Oracle Questioning in Active Learning
• Authors: Dani Kiyasseh, Tingting Zhu, David A. Clifton
• Abstract summary: We propose SoQal, a questioning strategy that determines when a label should be requested from an oracle. We perform experiments on five publically-available datasets and illustrate SoQal's superiority relative to baseline approaches.
• Score: 17.58391771585294
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Large sets of unlabelled data within the healthcare domain remain underutilized. Active learning offers a way to exploit these datasets by iteratively requesting an oracle (e.g. medical professional) to label instances. This process, which can be costly and time-consuming is overly-dependent upon an oracle. To alleviate this burden, we propose SoQal, a questioning strategy that dynamically determines when a label should be requested from an oracle. We perform experiments on five publically-available datasets and illustrate SoQal's superiority relative to baseline approaches, including its ability to reduce oracle label requests by up to 35%. SoQal also performs competitively in the presence of label noise: a scenario that simulates clinicians' uncertain diagnoses when faced with difficult classification tasks.

Related papers

• Active Generalized Category Discovery [60.69060965936214]
  Generalized Category Discovery (GCD) endeavors to cluster unlabeled samples from both novel and old classes. We take the spirit of active learning and propose a new setting called Active Generalized Category Discovery (AGCD) Our method achieves state-of-the-art performance on both generic and fine-grained datasets.
  arXiv  Detail & Related papers  (2024-03-07T07:12:24Z)
• Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language [10.47685316733524]
  We propose a novel method for interacting with the oracle in a selective and efficient way, using a retrieval-based approach. We use a neural network to encode the current state of the agent and the oracle, and retrieve the most relevant question from the corpus to ask the oracle. We show that our method can significantly improve the efficiency of RL by reducing the number of interactions needed to reach a certain level of performance.
  arXiv  Detail & Related papers  (2023-09-21T03:25:35Z)
• Text Summarization with Oracle Expectation [88.39032981994535]
  Extractive summarization produces summaries by identifying and concatenating the most important sentences in a document. Most summarization datasets do not come with gold labels indicating whether document sentences are summary-worthy. We propose a simple yet effective labeling algorithm that creates soft, expectation-based sentence labels.
  arXiv  Detail & Related papers  (2022-09-26T14:10:08Z)
• Information Gain Propagation: a new way to Graph Active Learning with Soft Labels [26.20597165750861]
  Graph Neural Networks (GNNs) have achieved great success in various tasks, but their performance highly relies on a large number of labeled nodes. We propose GNN-based Active Learning (AL) methods to improve the labeling efficiency by selecting the most valuable nodes to label. Our method significantly outperforms the state-of-the-art GNN-based AL methods in terms of both accuracy and labeling cost.
  arXiv  Detail & Related papers  (2022-03-02T13:28:25Z)
• QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction [57.56700153507383]
  This paper proposes a unified query attribute value extraction system in e-commerce search named QUEACO. For the NER phase, QUEACO adopts a novel teacher-student network, where a teacher network that is trained on the strongly-labeled data generates pseudo-labels. For the AVN phase, we also leverage the weakly-labeled query-to-attribute behavior data to normalize surface form attribute values from queries into canonical forms from products.
  arXiv  Detail & Related papers  (2021-08-19T03:24:23Z)
• KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction [111.74812895391672]
  We propose a Knowledge-aware Prompt-tuning approach with synergistic optimization (KnowPrompt) We inject latent knowledge contained in relation labels into prompt construction with learnable virtual type words and answer words.
  arXiv  Detail & Related papers  (2021-04-15T17:57:43Z)
• A Second-Order Approach to Learning with Instance-Dependent Label Noise [58.555527517928596]
  The presence of label noise often misleads the training of deep neural networks. We show that the errors in human-annotated labels are more likely to be dependent on the difficulty levels of tasks.
  arXiv  Detail & Related papers  (2020-12-22T06:36:58Z)
• PAL : Pretext-based Active Learning [2.869739951301252]
  We propose an active learning technique for deep neural networks that is more robust to mislabeling than the previously proposed techniques. We use a separate network to score the unlabeled samples for selection. The resultant technique also produces competitive accuracy in the absence of label noise.
  arXiv  Detail & Related papers  (2020-10-29T21:16:37Z)
• Deep Active Learning via Open Set Recognition [0.0]
  In many applications, data is easy to acquire but expensive and time-consuming to label prominent examples. We formulate active learning as an open-set recognition problem. Unlike current active learning methods, our algorithm can learn tasks without the need for task labels.
  arXiv  Detail & Related papers  (2020-07-04T22:09:17Z)
• ClarQ: A large-scale and diverse dataset for Clarification Question Generation [67.1162903046619]
  We devise a novel bootstrapping framework that assists in the creation of a diverse, large-scale dataset of clarification questions based on postcomments extracted from stackexchange. We quantitatively demonstrate the utility of the newly created dataset by applying it to the downstream task of question-answering. We release this dataset in order to foster research into the field of clarification question generation with the larger goal of enhancing dialog and question answering systems.
  arXiv  Detail & Related papers  (2020-06-10T17:56:50Z)
• SoQal: Selective Oracle Questioning for Consistency Based Active Learning of Cardiac Signals [17.58391771585294]
  Clinical settings are often characterized by abundant unlabelled data and limited labelled data. One way to mitigate this burden is via active learning (AL) which involves the (a) acquisition and (b) annotation of informative unlabelled instances. We show that BALC can outperform start-of-the-art acquisition functions such as BALD, and SoQal outperforms baseline methods even in the presence of a noisy oracle.
  arXiv  Detail & Related papers  (2020-04-20T18:20:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.