ActiveRAG: Revealing the Treasures of Knowledge via Active Learning
- URL: http://arxiv.org/abs/2402.13547v1
- Date: Wed, 21 Feb 2024 06:04:53 GMT
- Title: ActiveRAG: Revealing the Treasures of Knowledge via Active Learning
- Authors: Zhipeng Xu, Zhenghao Liu, Yibin Liu, Chenyan Xiong, Yukun Yan, Shuo
Wang, Shi Yu, Zhiyuan Liu, Ge Yu
- Abstract summary: Retrieval Augmented Generation (RAG) has introduced a new paradigm for Large Language Models (LLMs)
We present ActiveRAG, an innovative RAG framework that shifts from passive knowledge acquisition to an active learning mechanism.
Our experimental results demonstrate that ActiveRAG surpasses previous RAG models, achieving a 5% improvement on question-answering datasets.
- Score: 48.27288876691973
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Retrieval Augmented Generation (RAG) has introduced a new paradigm for Large
Language Models (LLMs), aiding in the resolution of knowledge-intensive tasks.
However, current RAG models position LLMs as passive knowledge receptors,
thereby restricting their capacity for learning and comprehending external
knowledge. In this paper, we present ActiveRAG, an innovative RAG framework
that shifts from passive knowledge acquisition to an active learning mechanism.
This approach utilizes the Knowledge Construction mechanism to develop a deeper
understanding of external knowledge by associating it with previously acquired
or memorized knowledge. Subsequently, it designs the Cognitive Nexus mechanism
to incorporate the outcomes from both chains of thought and knowledge
construction, thereby calibrating the intrinsic cognition of LLMs. Our
experimental results demonstrate that ActiveRAG surpasses previous RAG models,
achieving a 5% improvement on question-answering datasets. All data and codes
are available at https://github.com/OpenMatch/ActiveRAG.
Related papers
- Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language Models by Learning from Knowledge Graphs [55.317267269115845]
Chain-of-Knowledge (CoK) is a comprehensive framework for knowledge reasoning.
CoK includes methodologies for both dataset construction and model learning.
We conduct extensive experiments with KnowReason.
arXiv Detail & Related papers (2024-06-30T10:49:32Z) - KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning [32.086825891769585]
Knowledge-enhanced pre-trained language models (KEPLMs) leverage relation triples from knowledge graphs (KGs)
Previous works treat knowledge enhancement as two independent operations, i.e., knowledge injection and knowledge integration.
This paper jointly addresses the problems of detecting positions for knowledge injection and integrating external knowledge into the model in order to avoid injecting inaccurate or irrelevant knowledge.
arXiv Detail & Related papers (2024-06-24T07:32:35Z) - KG-RAG: Bridging the Gap Between Knowledge and Creativity [0.0]
Large Language Model Agents (LMAs) face issues such as information hallucinations, catastrophic forgetting, and limitations in processing long contexts.
This paper introduces a KG-RAG (Knowledge Graph-Retrieval Augmented Generation) pipeline to enhance the knowledge capabilities of LMAs.
Preliminary experiments on the ComplexWebQuestions dataset demonstrate notable improvements in the reduction of hallucinated content.
arXiv Detail & Related papers (2024-05-20T14:03:05Z) - A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models [71.25225058845324]
Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation.
Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge.
RA-LLMs have emerged to harness external and authoritative knowledge bases, rather than relying on the model's internal knowledge.
arXiv Detail & Related papers (2024-05-10T02:48:45Z) - REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain
Question Answering [122.62012375722124]
In existing methods, large language models (LLMs) cannot precisely assess the relevance of retrieved documents.
We propose REAR, a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA)
arXiv Detail & Related papers (2024-02-27T13:22:51Z) - ARKS: Active Retrieval in Knowledge Soup for Code Generation [18.22108704150575]
We introduce Active Retrieval in Knowledge Soup (ARKS), an advanced strategy for generalizing large language models for code.
We employ an active retrieval strategy that iteratively refines the query and updates the knowledge soup.
Experimental results on ChatGPT and CodeLlama demonstrate a substantial improvement in the average execution accuracy of ARKS on LLMs.
arXiv Detail & Related papers (2024-02-19T17:37:28Z) - Retrieval-Augmented Generation for Large Language Models: A Survey [17.82361213043507]
Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination.
Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases.
arXiv Detail & Related papers (2023-12-18T07:47:33Z) - Beyond Factuality: A Comprehensive Evaluation of Large Language Models
as Knowledge Generators [78.63553017938911]
Large language models (LLMs) outperform information retrieval techniques for downstream knowledge-intensive tasks.
However, community concerns abound regarding the factuality and potential implications of using this uncensored knowledge.
We introduce CONNER, designed to evaluate generated knowledge from six important perspectives.
arXiv Detail & Related papers (2023-10-11T08:22:37Z) - Towards a Universal Continuous Knowledge Base [49.95342223987143]
We propose a method for building a continuous knowledge base that can store knowledge imported from multiple neural networks.
Experiments on text classification show promising results.
We import the knowledge from multiple models to the knowledge base, from which the fused knowledge is exported back to a single model.
arXiv Detail & Related papers (2020-12-25T12:27:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.