Improving Low-Resource Question Answering using Active Learning in
Multiple Stages
- URL: http://arxiv.org/abs/2211.14880v1
- Date: Sun, 27 Nov 2022 16:31:33 GMT
- Title: Improving Low-Resource Question Answering using Active Learning in
Multiple Stages
- Authors: Maximilian Schmidt, Andrea Bartezzaghi, Jasmina Bogojeska, A.
Cristiano I. Malossi, Thang Vu
- Abstract summary: We propose a novel approach that combines data augmentation via question-answer generation with Active Learning to improve performance in low resource settings.
Our findings show that our novel approach, where humans are incorporated as early as possible in the process, boosts performance in the low-resource, domain-specific setting.
- Score: 5.238195443186652
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Neural approaches have become very popular in the domain of Question
Answering, however they require a large amount of annotated data. Furthermore,
they often yield very good performance but only in the domain they were trained
on. In this work we propose a novel approach that combines data augmentation
via question-answer generation with Active Learning to improve performance in
low resource settings, where the target domains are diverse in terms of
difficulty and similarity to the source domain. We also investigate Active
Learning for question answering in different stages, overall reducing the
annotation effort of humans. For this purpose, we consider target domains in
realistic settings, with an extremely low amount of annotated samples but with
many unlabeled documents, which we assume can be obtained with little effort.
Additionally, we assume sufficient amount of labeled data from the source
domain is available. We perform extensive experiments to find the best setup
for incorporating domain experts. Our findings show that our novel approach,
where humans are incorporated as early as possible in the process, boosts
performance in the low-resource, domain-specific setting, allowing for
low-labeling-effort question answering systems in new, specialized domains.
They further demonstrate how human annotation affects the performance of QA
depending on the stage it is performed.
Related papers
- Learning Feature Decomposition for Domain Adaptive Monocular Depth
Estimation [51.15061013818216]
Supervised approaches have led to great success with the advance of deep learning, but they rely on large quantities of ground-truth depth annotations.
Unsupervised domain adaptation (UDA) transfers knowledge from labeled source data to unlabeled target data, so as to relax the constraint of supervised learning.
We propose a novel UDA method for MDE, referred to as Learning Feature Decomposition for Adaptation (LFDA), which learns to decompose the feature space into content and style components.
arXiv Detail & Related papers (2022-07-30T08:05:35Z) - Crowdsourcing Learning as Domain Adaptation: A Case Study on Named
Entity Recognition [19.379850806513232]
We take a different point in this work, regarding all crowdsourced annotations as gold-standard with respect to the individual annotators.
We find that crowdsourcing could be highly similar to domain adaptation, and then the recent advances of cross-domain methods can be almost directly applied to crowdsourcing.
We investigate both unsupervised and supervised crowdsourcing learning, assuming that no or only small-scale expert annotations are available.
arXiv Detail & Related papers (2021-05-31T14:11:08Z) - Inferring Latent Domains for Unsupervised Deep Domain Adaptation [54.963823285456925]
Unsupervised Domain Adaptation (UDA) refers to the problem of learning a model in a target domain where labeled data are not available.
This paper introduces a novel deep architecture which addresses the problem of UDA by automatically discovering latent domains in visual datasets.
We evaluate our approach on publicly available benchmarks, showing that it outperforms state-of-the-art domain adaptation methods.
arXiv Detail & Related papers (2021-03-25T14:33:33Z) - Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic
Parsing [85.35582118010608]
Task-oriented semantic parsing is a critical component of virtual assistants.
Recent advances in deep learning have enabled several approaches to successfully parse more complex queries.
We propose a novel method that outperforms a supervised neural model at a 10-fold data reduction.
arXiv Detail & Related papers (2020-10-07T17:47:53Z) - A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [81.07994783143533]
Large-scale labeled training datasets have enabled deep neural networks to excel across a wide range of benchmark vision tasks.
In many applications, it is prohibitively expensive and time-consuming to obtain large quantities of labeled data.
To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
arXiv Detail & Related papers (2020-09-01T00:06:50Z) - Learning to Cluster under Domain Shift [20.00056591000625]
In this work we address the problem of transferring knowledge from a source to a target domain when both source and target data have no annotations.
Inspired by recent works on deep clustering, our approach leverages information from data gathered from multiple source domains.
We show that our method is able to automatically discover relevant semantic information even in presence of few target samples.
arXiv Detail & Related papers (2020-08-11T12:03:01Z) - Domain Adaptation for Semantic Parsing [68.81787666086554]
We propose a novel semantic for domain adaptation, where we have much fewer annotated data in the target domain compared to the source domain.
Our semantic benefits from a two-stage coarse-to-fine framework, thus can provide different and accurate treatments for the two stages.
Experiments on a benchmark dataset show that our method consistently outperforms several popular domain adaptation strategies.
arXiv Detail & Related papers (2020-06-23T14:47:41Z) - Latent Domain Learning with Dynamic Residual Adapters [26.018759356470767]
A practical shortcoming of deep neural networks is their specialization to a single task and domain.
Here we focus on a less explored, but more realistic case: learning from data from multiple domains, without access to domain annotations.
We address this limitation via dynamic residual adapters, an adaptive gating mechanism that helps account for latent domains.
arXiv Detail & Related papers (2020-06-01T15:00:11Z) - Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog [70.79442700890843]
We propose a novel Dynamic Fusion Network (DF-Net) which automatically exploit the relevance between the target domain and each domain.
With little training data, we show its transferability by outperforming prior best model by 13.9% on average.
arXiv Detail & Related papers (2020-04-23T08:17:22Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.