Exploring the Practicality of Generative Retrieval on Dynamic Corpora

• URL: http://arxiv.org/abs/2305.18952v5
• Date: Sat, 05 Oct 2024 08:36:04 GMT
• Title: Exploring the Practicality of Generative Retrieval on Dynamic Corpora
• Authors: Chaeeun Kim, Soyoung Yoon, Hyunji Lee, Joel Jang, Sohee Yang, Minjoon Seo,
• Abstract summary: In this paper, we focus on Generative Retrievals (GR), which apply autoregressive language models to IR problems. Our results on the StreamingQA benchmark demonstrate that GR is more adaptable to evolving knowledge (4-11%), robust in learning knowledge with temporal information, and efficient in terms of FLOPs (x6), indexing time (x6), and storage footprint (x4) Our paper highlights the potential of GR for future use in practical IR systems within dynamic environments.
• Score: 41.223804434693875
• License:
• Abstract: Benchmarking the performance of information retrieval (IR) is mostly conducted with a fixed set of documents (static corpora). However, in realistic scenarios, this is rarely the case and the documents to be retrieved are constantly updated and added. In this paper, we focus on Generative Retrievals (GR), which apply autoregressive language models to IR problems, and explore their adaptability and robustness in dynamic scenarios. We also conduct an extensive evaluation of computational and memory efficiency, crucial factors for real-world deployment of IR systems handling vast and ever-changing document collections. Our results on the StreamingQA benchmark demonstrate that GR is more adaptable to evolving knowledge (4-11%), robust in learning knowledge with temporal information, and efficient in terms of inference FLOPs (x2), indexing time (x6), and storage footprint (x4) compared to Dual Encoders (DE), which are commonly used in retrieval systems. Our paper highlights the potential of GR for future use in practical IR systems within dynamic environments.

Related papers

• Developing Retrieval Augmented Generation (RAG) based LLM Systems from PDFs: An Experience Report [3.4632900249241874]
  This paper presents an experience report on the development of Retrieval Augmented Generation (RAG) systems using PDF documents as the primary data source. The RAG architecture combines generative capabilities of Large Language Models (LLMs) with the precision of information retrieval. The practical implications of this research lie in enhancing the reliability of generative AI systems in various sectors.
  arXiv  Detail & Related papers  (2024-10-21T12:21:49Z)
• Retriever-and-Memory: Towards Adaptive Note-Enhanced Retrieval-Augmented Generation [72.70046559930555]
  We propose a generic RAG approach called Adaptive Note-Enhanced RAG (Adaptive-Note) for complex QA tasks. Specifically, Adaptive-Note introduces an overarching view of knowledge growth, iteratively gathering new information in the form of notes. In addition, we employ an adaptive, note-based stop-exploration strategy to decide "what to retrieve and when to stop" to encourage sufficient knowledge exploration.
  arXiv  Detail & Related papers  (2024-10-11T14:03:29Z)
• LightRAG: Simple and Fast Retrieval-Augmented Generation [12.86888202297654]
  Retrieval-Augmented Generation (RAG) systems enhance large language models (LLMs) by integrating external knowledge sources. Existing RAG systems have significant limitations, including reliance on flat data representations and inadequate contextual awareness. We propose LightRAG, which incorporates graph structures into text indexing and retrieval processes.
  arXiv  Detail & Related papers  (2024-10-08T08:00:12Z)
• From Matching to Generation: A Survey on Generative Information Retrieval [21.56093567336119]
  generative information retrieval (GenIR) has emerged as a novel paradigm, gaining increasing attention in recent years. This paper aims to systematically review the latest research progress in GenIR.
  arXiv  Detail & Related papers  (2024-04-23T09:05:37Z)
• CorpusLM: Towards a Unified Language Model on Corpus for Knowledge-Intensive Tasks [20.390672895839757]
  Retrieval-augmented generation (RAG) has emerged as a popular solution to enhance factual accuracy. Traditional retrieval modules often rely on large document index and disconnect with generative tasks. We propose textbfCorpusLM, a unified language model that integrates generative retrieval, closed-book generation, and RAG.
  arXiv  Detail & Related papers  (2024-02-02T06:44:22Z)
• Analysis of the Memorization and Generalization Capabilities of AI Agents: Are Continual Learners Robust? [91.682459306359]
  In continual learning (CL), an AI agent learns from non-stationary data streams under dynamic environments. In this paper, a novel CL framework is proposed to achieve robust generalization to dynamic environments while retaining past knowledge. The generalization and memorization performance of the proposed framework are theoretically analyzed.
  arXiv  Detail & Related papers  (2023-09-18T21:00:01Z)
• Building Interpretable and Reliable Open Information Retriever for New Domains Overnight [67.03842581848299]
  Information retrieval is a critical component for many down-stream tasks such as open-domain question answering (QA) We propose an information retrieval pipeline that uses entity/event linking model and query decomposition model to focus more accurately on different information units of the query. We show that, while being more interpretable and reliable, our proposed pipeline significantly improves passage coverages and denotation accuracies across five IR and QA benchmarks.
  arXiv  Detail & Related papers  (2023-08-09T07:47:17Z)
• Synergistic Interplay between Search and Large Language Models for Information Retrieval [141.18083677333848]
  InteR allows RMs to expand knowledge in queries using LLM-generated knowledge collections. InteR achieves overall superior zero-shot retrieval performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-12T11:58:15Z)
• Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking [56.80065604034095]
  We introduce a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. To evaluate our different integration strategies, we transform four existing information retrieval datasets into the relevance feedback scenario.
  arXiv  Detail & Related papers  (2022-10-19T16:19:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.