REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain
Question Answering
- URL: http://arxiv.org/abs/2402.17497v1
- Date: Tue, 27 Feb 2024 13:22:51 GMT
- Title: REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain
Question Answering
- Authors: Yuhao Wang, Ruiyang Ren, Junyi Li, Wayne Xin Zhao, Jing Liu, Ji-Rong
Wen
- Abstract summary: In existing methods, large language models (LLMs) cannot precisely assess the relevance of retrieved documents.
We propose REAR, a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA)
- Score: 122.62012375722124
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Considering the limited internal parametric knowledge, retrieval-augmented
generation (RAG) has been widely used to extend the knowledge scope of large
language models (LLMs). Despite the extensive efforts on RAG research, in
existing methods, LLMs cannot precisely assess the relevance of retrieved
documents, thus likely leading to misleading or even incorrect utilization of
external knowledge (i.e., retrieved documents). To address this issue, in this
paper, we propose REAR, a RElevance-Aware Retrieval-augmented approach for
open-domain question answering (QA). As the key motivation, we aim to enhance
the self-awareness of source relevance for LLMs, so as to adaptively utilize
external knowledge in RAG systems. Specially, we develop a new architecture for
LLM based RAG system, by incorporating a specially designed rank head that
precisely assesses the relevance of retrieved documents. Furthermore, we
propose an improved training method based on bi-granularity relevance fusion
and noise-resistant training. By combining the improvements in both
architecture and training, our proposed REAR can better utilize external
knowledge by effectively perceiving the relevance of retrieved documents.
Experiments on four open-domain QA tasks show that REAR significantly
outperforms previous a number of competitive RAG approaches. Our code and data
can be accessed at https://github.com/RUCAIBox/REAR.
Related papers
- Enhancing Retrieval and Managing Retrieval: A Four-Module Synergy for Improved Quality and Efficiency in RAG Systems [14.62114319247837]
Retrieval-augmented generation (RAG) techniques leverage the in-context learning capabilities of large language models (LLMs) to produce more accurate and relevant responses.
A critical component, the Query Rewriter module, enhances knowledge retrieval by generating a search-friendly query.
These four RAG modules synergistically improve the response quality and efficiency of the RAG system.
arXiv Detail & Related papers (2024-07-15T12:35:00Z) - A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation.
Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge.
RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
arXiv Detail & Related papers (2024-05-10T02:48:45Z) - The Power of Noise: Redefining Retrieval for RAG Systems [19.387105120040157]
Retrieval-Augmented Generation (RAG) has emerged as a method to extend beyond the pre-trained knowledge of Large Language Models.
We focus on the type of passages IR systems within a RAG solution should retrieve.
arXiv Detail & Related papers (2024-01-26T14:14:59Z) - Merging Generated and Retrieved Knowledge for Open-Domain QA [72.42262579925911]
COMBO is a compatibility-Oriented knowledge Merging for Better Open-domain QA framework.
We show that COMBO outperforms competitive baselines on three out of four tested open-domain QA benchmarks.
arXiv Detail & Related papers (2023-10-22T19:37:06Z) - Self-Knowledge Guided Retrieval Augmentation for Large Language Models [59.771098292611846]
Large language models (LLMs) have shown superior performance without task-specific fine-tuning.
Retrieval-based methods can offer non-parametric world knowledge and improve the performance on tasks such as question answering.
Self-Knowledge guided Retrieval augmentation (SKR) is a simple yet effective method which can let LLMs refer to the questions they have previously encountered.
arXiv Detail & Related papers (2023-10-08T04:22:33Z) - Building Interpretable and Reliable Open Information Retriever for New
Domains Overnight [67.03842581848299]
Information retrieval is a critical component for many down-stream tasks such as open-domain question answering (QA)
We propose an information retrieval pipeline that uses entity/event linking model and query decomposition model to focus more accurately on different information units of the query.
We show that, while being more interpretable and reliable, our proposed pipeline significantly improves passage coverages and denotation accuracies across five IR and QA benchmarks.
arXiv Detail & Related papers (2023-08-09T07:47:17Z) - Synergistic Interplay between Search and Large Language Models for
Information Retrieval [141.18083677333848]
InteR allows RMs to expand knowledge in queries using LLM-generated knowledge collections.
InteR achieves overall superior zero-shot retrieval performance compared to state-of-the-art methods.
arXiv Detail & Related papers (2023-05-12T11:58:15Z) - Improving the Domain Adaptation of Retrieval Augmented Generation (RAG)
Models for Open Domain Question Answering [9.404960572390852]
Retrieval Augment Generation (RAG) is a recent advancement in Open-Domain Question Answering (ODQA)
RAG has only been trained and explored with a Wikipedia-based external knowledge base.
We propose textitRAG-end2end, an extension to RAG, that can adapt to a domain-specific knowledge base.
arXiv Detail & Related papers (2022-10-06T01:21:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.