In Defense of RAG in the Era of Long-Context Language Models

• URL: http://arxiv.org/abs/2409.01666v1
• Date: Tue, 3 Sep 2024 07:17:41 GMT
• Title: In Defense of RAG in the Era of Long-Context Language Models
• Authors: Tan Yu, Anbang Xu, Rama Akkiraju,
• Abstract summary: Retrieval-augmented generation (RAG) has been a reliable solution for context-based answer generation in the past. Recent studies show that long-context LLMs significantly outperform RAG in long-context applications. We propose an order-preserve retrieval-augmented generation (OP-RAG) mechanism, which significantly improves the performance of RAG for long-context question-answer applications.
• Score: 17.397639724806364
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Overcoming the limited context limitations in early-generation LLMs, retrieval-augmented generation (RAG) has been a reliable solution for context-based answer generation in the past. Recently, the emergence of long-context LLMs allows the models to incorporate much longer text sequences, making RAG less attractive. Recent studies show that long-context LLMs significantly outperform RAG in long-context applications. Unlike the existing works favoring the long-context LLM over RAG, we argue that the extremely long context in LLMs suffers from a diminished focus on relevant information and leads to potential degradation in answer quality. This paper revisits the RAG in long-context answer generation. We propose an order-preserve retrieval-augmented generation (OP-RAG) mechanism, which significantly improves the performance of RAG for long-context question-answer applications. With OP-RAG, as the number of retrieved chunks increases, the answer quality initially rises, and then declines, forming an inverted U-shaped curve. There exist sweet points where OP-RAG could achieve higher answer quality with much less tokens than long-context LLM taking the whole context as input. Extensive experiments on public benchmark demonstrate the superiority of our OP-RAG.

Related papers

• Long Context RAG Performance of Large Language Models [29.7557824450885]
  Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) This paper presents a study of the impact of increased context length on RAG performance across 20 popular open source and commercial LLMs.
  arXiv  Detail & Related papers  (2024-11-05T22:37:43Z)
• What is Wrong with Perplexity for Long-context Language Modeling? [71.34933096461124]
  Long-context inputs are crucial for large language models (LLMs) in tasks such as extended conversations, document summarization, and many-shot in-context learning. Perplexity (PPL) has proven unreliable for assessing long-context capabilities. We propose bfLongPPL, a novel metric that focuses on key tokens by employing a long-short context contrastive method to identify them.
  arXiv  Detail & Related papers  (2024-10-31T09:39:28Z)
• LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering [27.114593394058144]
  LongRAG is a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA. LongRAG significantly outperforms long-context LLMs (up by 6.94%), advanced RAG (up by 6.16%), and Vanilla RAG (up by 17.25%)
  arXiv  Detail & Related papers  (2024-10-23T17:24:58Z)
• Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs [12.878608250420832]
  We propose textitgraph of records (textbfGoR) to enhance RAG for long-context global summarization. Inspired by the textitretrieve-then-generate paradigm of RAG, we construct a graph by establishing an edge between the retrieved text chunks and the corresponding LLM-generated response. To further uncover the intricate correlations between them, GoR features a textitgraph neural network and an elaborately designed textitBERTScore-based objective for self-supervised model training.
  arXiv  Detail & Related papers  (2024-10-14T18:34:29Z)
• ALR$^2$: A Retrieve-then-Reason Framework for Long-context Question Answering [42.146660039671076]
  We develop a retrieve-then-reason framework for large language models (LLMs) We find that modern LLMs struggle to accurately retrieve relevant facts and instead, often hallucinate "retrieved facts" We introduce ALR$2$, a method that augments the long-context reasoning capability of LLMs via an explicit two-stage procedure.
  arXiv  Detail & Related papers  (2024-10-04T08:29:12Z)
• Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach [26.02167477129771]
  Retrieval Augmented Generation (RAG) has been a powerful tool for Large Language Models (LLMs) to efficiently process overly lengthy contexts. We compare RAG and long-context (LC) LLMs, aiming to leverage the strengths of both. We propose Self-Route, a simple yet effective method that routes queries to RAG or LC based on model self-reflection.
  arXiv  Detail & Related papers  (2024-07-23T20:51:52Z)
• RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answering [61.19126689470398]
  Long-form RobustQA (LFRQA) is a new dataset covering 26K queries and large corpora across seven different domains. We show via experiments that RAG-QA Arena and human judgments on answer quality are highly correlated. Only 41.3% of the most competitive LLM's answers are preferred to LFRQA's answers, demonstrating RAG-QA Arena as a challenging evaluation platform for future research.
  arXiv  Detail & Related papers  (2024-07-19T03:02:51Z)
• Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting [68.90949377014742]
  Speculative RAG is a framework that leverages a larger generalist LM to efficiently verify multiple RAG drafts produced in parallel by a smaller, distilled specialist LM. Our method accelerates RAG by delegating drafting to the smaller specialist LM, with the larger generalist LM performing a single verification pass over the drafts. It notably enhances accuracy by up to 12.97% while reducing latency by 51% compared to conventional RAG systems on PubHealth.
  arXiv  Detail & Related papers  (2024-07-11T06:50:19Z)
• Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA [71.04146366608904]
  Long-context modeling capabilities have garnered widespread attention, leading to the emergence of Large Language Models (LLMs) with ultra-context windows. We propose a novel long-context benchmark, Loong, aligning with realistic scenarios through extended multi-document question answering (QA) Loong introduces four types of tasks with a range of context lengths: Spotlight Locating, Comparison, Clustering, and Chain of Reasoning.
  arXiv  Detail & Related papers  (2024-06-25T09:42:56Z)
• Unsupervised Information Refinement Training of Large Language Models for Retrieval-Augmented Generation [128.01050030936028]
  We propose an information refinement training method named InFO-RAG. InFO-RAG is low-cost and general across various tasks. It improves the performance of LLaMA2 by an average of 9.39% relative points.
  arXiv  Detail & Related papers  (2024-02-28T08:24:38Z)
• Benchmarking Large Language Models in Retrieval-Augmented Generation [53.504471079548]
  We systematically investigate the impact of Retrieval-Augmented Generation on large language models. We analyze the performance of different large language models in 4 fundamental abilities required for RAG. We establish Retrieval-Augmented Generation Benchmark (RGB), a new corpus for RAG evaluation in both English and Chinese.
  arXiv  Detail & Related papers  (2023-09-04T08:28:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.