Generate rather than Retrieve: Large Language Models are Strong Context Generators

• URL: http://arxiv.org/abs/2209.10063v1
• Date: Wed, 21 Sep 2022 01:30:59 GMT
• Title: Generate rather than Retrieve: Large Language Models are Strong Context Generators
• Authors: Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
• Abstract summary: We present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer.
• Score: 74.87021992611672
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge-intensive tasks, such as open-domain question answering (QA), require access to a large amount of world or domain knowledge. A common approach for knowledge-intensive tasks is to employ a retrieve-then-read pipeline that first retrieves a handful of relevant contextual documents from an external corpus such as Wikipedia and then predicts an answer conditioned on the retrieved documents. In this paper, we present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer. Furthermore, we propose a novel clustering-based prompting method that selects distinct prompts, resulting in the generated documents that cover different perspectives, leading to better recall over acceptable answers. We conduct extensive experiments on three different knowledge-intensive tasks, including open-domain QA, fact checking, and dialogue system. Notably, GenRead achieves 71.6 and 54.4 exact match scores on TriviaQA and WebQ, significantly outperforming the state-of-the-art retrieve-then-read pipeline DPR-FiD by +4.0 and +3.9, without retrieving any documents from any external knowledge source. Lastly, we demonstrate the model performance can be further improved by combining retrieval and generation.

Related papers

• Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA [51.3033125256716]
  We model the subgraph retrieval task as a conditional generation task handled by small language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, competitive retrieval performance compared to state-of-the-art models. Our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks.
  arXiv  Detail & Related papers  (2024-10-08T15:22:36Z)
• Exploring Hint Generation Approaches in Open-Domain Question Answering [16.434748534272014]
  We introduce a novel context preparation approach called HINTQA. Unlike traditional methods, HINTQA prompts LLMs to produce hints about potential answers for the question. We demonstrate that hints enhance the accuracy of answers more than retrieved and generated contexts.
  arXiv  Detail & Related papers  (2024-09-24T13:50:32Z)
• Harnessing Multi-Role Capabilities of Large Language Models for Open-Domain Question Answering [40.2758450304531]
  Open-domain question answering (ODQA) has emerged as a pivotal research spotlight in information systems. We propose a framework that formulates the ODQA process into three basic steps: query expansion, document selection, and answer generation. We introduce a novel prompt optimization algorithm to refine role-playing prompts and steer LLMs to produce higher-quality evidence and answers.
  arXiv  Detail & Related papers  (2024-03-08T11:09:13Z)
• UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models [22.457013726785295]
  We present textbfUniGen, a textbfUnified textbfGenerative framework for retrieval and question answering. UniGen integrates both tasks into a single generative model leveraging the capabilities of large language models.
  arXiv  Detail & Related papers  (2023-12-18T09:13:41Z)
• Retrieval-Generation Synergy Augmented Large Language Models [30.53260173572783]
  We propose an iterative retrieval-generation collaborative framework. We conduct experiments on four question answering datasets, including single-hop QA and multi-hop QA tasks.
  arXiv  Detail & Related papers  (2023-10-08T12:50:57Z)
• Generator-Retriever-Generator Approach for Open-Domain Question Answering [18.950517545413813]
  We propose a novel approach that combines document retrieval techniques with a large language model (LLM) In parallel, a dual-encoder network retrieves documents that are relevant to the question from an external corpus. GRG outperforms the state-of-the-art generate-then-read and retrieve-then-read pipelines.
  arXiv  Detail & Related papers  (2023-07-21T00:34:38Z)
• Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy [164.83371924650294]
  We show that strong performance can be achieved by a method we call Iter-RetGen, which synergizes retrieval and generation in an iterative manner. A model output shows what might be needed to finish a task, and thus provides an informative context for retrieving more relevant knowledge. Iter-RetGen processes all retrieved knowledge as a whole and largely preserves the flexibility in generation without structural constraints.
  arXiv  Detail & Related papers  (2023-05-24T16:17:36Z)
• Active Retrieval Augmented Generation [123.68874416084499]
  Augmenting large language models (LMs) by retrieving information from external knowledge resources is one promising solution. Most existing retrieval augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. We propose Forward-Looking Active REtrieval augmented generation (FLARE), a generic method which iteratively uses a prediction of the upcoming sentence to anticipate future content.
  arXiv  Detail & Related papers  (2023-05-11T17:13:40Z)
• Recitation-Augmented Language Models [85.30591349383849]
  We show that RECITE is a powerful paradigm for knowledge-intensive NLP tasks. Specifically, we show that by utilizing recitation as the intermediate step, a recite-and-answer scheme can achieve new state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-10-04T00:49:20Z)
• Text Summarization with Latent Queries [60.468323530248945]
  We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms. Under a deep generative framework, our system jointly optimize a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time. Our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.
  arXiv  Detail & Related papers  (2021-05-31T21:14:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.