Active Learning Over Multiple Domains in Natural Language Tasks
- URL: http://arxiv.org/abs/2202.00254v1
- Date: Tue, 1 Feb 2022 07:27:18 GMT
- Title: Active Learning Over Multiple Domains in Natural Language Tasks
- Authors: Shayne Longpre, Julia Reisler, Edward Greg Huang, Yi Lu, Andrew Frank,
Nikhil Ramesh, Chris DuBois
- Abstract summary: We survey a variety of techniques in active learning (AL), domain shift detection (DS), and multi-domain sampling.
Among 18 acquisition functions from 4 families of methods, we find H- Divergence methods, and particularly our proposed variant DAL-E, yield effective results.
Our findings yield the first comprehensive analysis of both existing and novel methods for practitioners faced with multi-domain active learning for natural language tasks.
- Score: 6.8700723936987975
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Studies of active learning traditionally assume the target and source data
stem from a single domain. However, in realistic applications, practitioners
often require active learning with multiple sources of out-of-distribution
data, where it is unclear a priori which data sources will help or hurt the
target domain. We survey a wide variety of techniques in active learning (AL),
domain shift detection (DS), and multi-domain sampling to examine this
challenging setting for question answering and sentiment analysis. We ask (1)
what family of methods are effective for this task? And, (2) what properties of
selected examples and domains achieve strong results? Among 18 acquisition
functions from 4 families of methods, we find H- Divergence methods, and
particularly our proposed variant DAL-E, yield effective results, averaging
2-3% improvements over the random baseline. We also show the importance of a
diverse allocation of domains, as well as room-for-improvement of existing
methods on both domain and example selection. Our findings yield the first
comprehensive analysis of both existing and novel methods for practitioners
faced with multi-domain active learning for natural language tasks.
Related papers
- An Unsupervised Domain Adaptation Method for Locating Manipulated Region in partially fake Audio [43.91918688112442]
We propose an unsupervised method named Samples mining with Diversity and Entropy (SDE)
Our method first learns from a collection of diverse experts that achieve great performance from different perspectives in the source domain.
By introducing 10% of unknown samples from the target domain, we achieved an F1 score of 43.84%, which represents a relative increase of 77.2% compared to the second-best method.
arXiv Detail & Related papers (2024-07-11T07:32:16Z) - Domain Adversarial Active Learning for Domain Generalization
Classification [8.003401798449337]
Domain generalization models aim to learn cross-domain knowledge from source domain data, to improve performance on unknown target domains.
Recent research has demonstrated that diverse and rich source domain samples can enhance domain generalization capability.
We propose a domain-adversarial active learning (DAAL) algorithm for classification tasks in domain generalization.
arXiv Detail & Related papers (2024-03-10T10:59:22Z) - Revisiting the Domain Shift and Sample Uncertainty in Multi-source
Active Domain Transfer [69.82229895838577]
Active Domain Adaptation (ADA) aims to maximally boost model adaptation in a new target domain by actively selecting a limited number of target data to annotate.
This setting neglects the more practical scenario where training data are collected from multiple sources.
This motivates us to target a new and challenging setting of knowledge transfer that extends ADA from a single source domain to multiple source domains.
arXiv Detail & Related papers (2023-11-21T13:12:21Z) - Perturbation-Based Two-Stage Multi-Domain Active Learning [31.073745612552926]
We propose a perturbation-based two-stage multi-domain active learning (P2S-MDAL) method incorporated into the well-regarded ASP-MTL model.
P2S-MDAL involves allocating budgets for domains and establishing regions for diversity selection.
A perturbation metric has been introduced to evaluate the robustness of the shared feature extractor of the model.
arXiv Detail & Related papers (2023-06-19T04:58:32Z) - Label Distribution Learning for Generalizable Multi-source Person
Re-identification [48.77206888171507]
Person re-identification (Re-ID) is a critical technique in the video surveillance system.
It is difficult to directly apply the supervised model to arbitrary unseen domains.
We propose a novel label distribution learning (LDL) method to address the generalizable multi-source person Re-ID task.
arXiv Detail & Related papers (2022-04-12T15:59:10Z) - Self-Supervised Graph Neural Network for Multi-Source Domain Adaptation [51.21190751266442]
Domain adaptation (DA) tries to tackle the scenarios when the test data does not fully follow the same distribution of the training data.
By learning from large-scale unlabeled samples, self-supervised learning has now become a new trend in deep learning.
We propose a novel textbfSelf-textbfSupervised textbfGraph Neural Network (SSG) to enable more effective inter-task information exchange and knowledge sharing.
arXiv Detail & Related papers (2022-04-08T03:37:56Z) - Unsupervised Sentiment Analysis by Transferring Multi-source Knowledge [22.880509132587807]
We propose a two-stage domain adaptation framework for sentiment analysis.
In the first stage, a multi-task methodology-based shared private architecture is employed to explicitly model the domain common features.
In the second stage, two elaborate mechanisms are embedded in the shared private architecture to transfer knowledge from multiple source domains.
arXiv Detail & Related papers (2021-05-09T03:02:19Z) - Domain Adaptation for Learning Generator from Paired Few-Shot Data [72.04430033118426]
We propose a Paired Few-shot GAN (PFS-GAN) model for learning generators with sufficient source data and a few target data.
Our method has better quantitative and qualitative results on the generated target-domain data with higher diversity in comparison to several baselines.
arXiv Detail & Related papers (2021-02-25T10:11:44Z) - A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [81.07994783143533]
Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks.
In many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data.
To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
arXiv Detail & Related papers (2020-09-01T00:06:50Z) - Domain Adaption for Knowledge Tracing [65.86619804954283]
We propose a novel adaptable framework, namely knowledge tracing (AKT) to address the DAKT problem.
For the first aspect, we incorporate the educational characteristics (e.g., slip, guess, question texts) based on the deep knowledge tracing (DKT) to obtain a good performed knowledge tracing model.
For the second aspect, we propose and adopt three domain adaptation processes. First, we pre-train an auto-encoder to select useful source instances for target model training.
arXiv Detail & Related papers (2020-01-14T15:04:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.