Improving the Domain Adaptation of Retrieval Augmented Generation (RAG)
Models for Open Domain Question Answering
- URL: http://arxiv.org/abs/2210.02627v1
- Date: Thu, 6 Oct 2022 01:21:25 GMT
- Title: Improving the Domain Adaptation of Retrieval Augmented Generation (RAG)
Models for Open Domain Question Answering
- Authors: Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu
Kaluarachchi, Rajib Rana, Suranga Nanayakkara
- Abstract summary: Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain Question Answering (ODQA)
RAG has only been trained and explored with a Wikipedia-based external knowledge base.
We propose textitRAG-end2end, an extension to RAG, that can adapt to a domain-specific knowledge base.
- Score: 9.404960572390852
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain
Question Answering (ODQA). RAG has only been trained and explored with a
Wikipedia-based external knowledge base and is not optimized for use in other
specialized domains such as healthcare and news. In this paper, we evaluate the
impact of joint training of the retriever and generator components of RAG for
the task of domain adaptation in ODQA. We propose \textit{RAG-end2end}, an
extension to RAG, that can adapt to a domain-specific knowledge base by
updating all components of the external knowledge base during training. In
addition, we introduce an auxiliary training signal to inject more
domain-specific knowledge. This auxiliary signal forces \textit{RAG-end2end} to
reconstruct a given sentence by accessing the relevant information from the
external knowledge base. Our novel contribution is unlike RAG, RAG-end2end does
joint training of the retriever and generator for the end QA task and domain
adaptation. We evaluate our approach with datasets from three domains:
COVID-19, News, and Conversations, and achieve significant performance
improvements compared to the original RAG model. Our work has been open-sourced
through the Huggingface Transformers library, attesting to our work's
credibility and technical consistency.
Related papers
- DuetRAG: Collaborative Retrieval-Augmented Generation [57.440772556318926]
Collaborative Retrieval-Augmented Generation framework, DuetRAG, proposed.
bootstrapping philosophy is to simultaneously integrate the domain fintuning and RAG models.
arXiv Detail & Related papers (2024-05-12T09:48:28Z) - Retrieval-Augmented Generation for AI-Generated Content: A Survey [38.50754568320154]
Retrieval-Augmented Generation (RAG) has emerged as a paradigm to address such challenges.
RAG introduces the information retrieval process, which enhances the generation process by retrieving relevant objects from available data stores.
In this paper, we comprehensively review existing efforts that integrate RAG technique into AIGC scenarios.
arXiv Detail & Related papers (2024-02-29T18:59:01Z) - REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain
Question Answering [122.62012375722124]
In existing methods, large language models (LLMs) cannot precisely assess the relevance of retrieved documents.
We propose REAR, a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA)
arXiv Detail & Related papers (2024-02-27T13:22:51Z) - Quality > Quantity: Synthetic Corpora from Foundation Models for
Closed-Domain Extractive Question Answering [35.38140071573828]
We study extractive question answering within closed domains and introduce the concept of targeted pre-training.
Our proposed framework uses Galactica to generate synthetic, targeted'' corpora that align with specific writing styles and topics.
arXiv Detail & Related papers (2023-10-25T20:48:16Z) - Bidirectional Generative Framework for Cross-domain Aspect-based
Sentiment Analysis [68.742820522137]
Cross-domain aspect-based sentiment analysis (ABSA) aims to perform various fine-grained sentiment analysis tasks on a target domain by transferring knowledge from a source domain.
We propose a unified bidirectional generative framework to tackle various cross-domain ABSA tasks.
Our framework trains a generative model in both text-to-label and label-to-text directions.
arXiv Detail & Related papers (2023-05-16T15:02:23Z) - Domain Re-Modulation for Few-Shot Generative Domain Adaptation [71.47730150327818]
Generative Domain Adaptation (GDA) involves transferring a pre-trained generator from one domain to a new domain using only a few reference images.
Inspired by the way human brains acquire knowledge in new domains, we present an innovative generator structure called Domain Re-Modulation (DoRM)
DoRM not only meets the criteria of high quality, large synthesis diversity, and cross-domain consistency, but also incorporates memory and domain association.
arXiv Detail & Related papers (2023-02-06T03:55:35Z) - Adversarial Bi-Regressor Network for Domain Adaptive Regression [52.5168835502987]
It is essential to learn a cross-domain regressor to mitigate the domain shift.
This paper proposes a novel method Adversarial Bi-Regressor Network (ABRNet) to seek more effective cross-domain regression model.
arXiv Detail & Related papers (2022-09-20T18:38:28Z) - Unsupervised Domain Adaptation via Style-Aware Self-intermediate Domain [52.08301776698373]
Unsupervised domain adaptation (UDA) has attracted considerable attention, which transfers knowledge from a label-rich source domain to a related but unlabeled target domain.
We propose a novel style-aware feature fusion method (SAFF) to bridge the large domain gap and transfer knowledge while alleviating the loss of class-discnative information.
arXiv Detail & Related papers (2022-09-05T10:06:03Z) - FRIDA -- Generative Feature Replay for Incremental Domain Adaptation [34.00059350161178]
We propose a novel framework called Feature based Incremental Domain Adaptation (FRIDA)
For domain alignment, we propose a simple extension of the popular domain adversarial neural network (DANN) called DANN-IB.
Experiment results on Office-Home, Office-CalTech, and DomainNet datasets confirm that FRIDA maintains superior stability-plasticity trade-off than the literature.
arXiv Detail & Related papers (2021-12-28T22:24:32Z) - Sequential Domain Adaptation through Elastic Weight Consolidation for
Sentiment Analysis [3.1473798197405944]
We propose a model-independent framework - Sequential Domain Adaptation (SDA)
Our experiments show that the proposed framework enables simple architectures such as CNNs to outperform complex state-of-the-art models in domain adaptation of sentiment analysis (SA)
In addition, we observe that the effectiveness of a harder first Anti-Curriculum ordering of source domains leads to maximum performance.
arXiv Detail & Related papers (2020-07-02T15:21:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.