From Documents to Dialogue: Building KG-RAG Enhanced AI Assistants

• URL: http://arxiv.org/abs/2502.15237v1
• Date: Fri, 21 Feb 2025 06:22:12 GMT
• Title: From Documents to Dialogue: Building KG-RAG Enhanced AI Assistants
• Authors: Manisha Mukherjee, Sungchul Kim, Xiang Chen, Dan Luo, Tong Yu, Tung Mai,
• Abstract summary: We use a Retrieval-Augmented Generation (RAG) framework powered by a Knowledge Graph (KG) to retrieve relevant information from external knowledge sources.<n>Our KG-RAG system retrieves relevant provenances, which are added to the user prompts context before being sent to the LLM generating the response.<n>Our evaluation demonstrates that this approach significantly enhances response relevance, reducing irrelevant answers by over 50% and increasing fully relevant answers by 88% compared to the existing production system.
• Score: 28.149173430599525
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Adobe Experience Platform AI Assistant is a conversational tool that enables organizations to interact seamlessly with proprietary enterprise data through a chatbot. However, due to access restrictions, Large Language Models (LLMs) cannot retrieve these internal documents, limiting their ability to generate accurate zero-shot responses. To overcome this limitation, we use a Retrieval-Augmented Generation (RAG) framework powered by a Knowledge Graph (KG) to retrieve relevant information from external knowledge sources, enabling LLMs to answer questions over private or previously unseen document collections. In this paper, we propose a novel approach for building a high-quality, low-noise KG. We apply several techniques, including incremental entity resolution using seed concepts, similarity-based filtering to deduplicate entries, assigning confidence scores to entity-relation pairs to filter for high-confidence pairs, and linking facts to source documents for provenance. Our KG-RAG system retrieves relevant tuples, which are added to the user prompts context before being sent to the LLM generating the response. Our evaluation demonstrates that this approach significantly enhances response relevance, reducing irrelevant answers by over 50% and increasing fully relevant answers by 88% compared to the existing production system.

Related papers

• Oreo: A Plug-in Context Reconstructor to Enhance Retrieval-Augmented Generation [28.568010424711563]
  Large Language Models (LLMs) remain vulnerable to hallucinations due to their limited parametric knowledge and lack of domain-specific expertise.<n>Retrieval-Augmented Generation (RAG) addresses this challenge by incorporating external document retrieval to augment the knowledge base of LLMs.<n>We introduce a compact, efficient, and pluggable module designed to refine external knowledge sources before feeding them to the generator.
  arXiv  Detail & Related papers  (2025-02-18T16:38:39Z)
• QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance [1.433758865948252]
  This work presents a novel architecture for building Retrieval-Augmented Generation (RAG) systems. RAG architecture is constructed to generate responses from the target document. We introduce QuIM-RAG, a novel approach for the retrieval mechanism in our system.
  arXiv  Detail & Related papers  (2025-01-06T01:07:59Z)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation [81.18701211912779]
  We introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. Our method has achieved state-of-the-art performance on two common datasets.
  arXiv  Detail & Related papers  (2024-12-24T16:38:04Z)
• BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression [91.23933111083389]
  Retrieval-augmented generation (RAG) can supplement large language models (LLMs) by integrating external knowledge.<n>This paper presents BRIEF, a lightweight approach that performs query-aware multi-hop reasoning.<n>Based on our synthetic data built entirely by open-source models, BRIEF generates more concise summaries.
  arXiv  Detail & Related papers  (2024-10-20T04:24:16Z)
• CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning [0.9295048974480845]
  We propose CuriousLLM, an enhancement that integrates a curiosity-driven reasoning mechanism into an LLM agent.<n>This mechanism enables the agent to generate relevant follow-up questions, thereby guiding the information retrieval process more efficiently.<n>Our experiments show that CuriousLLM significantly boosts LLM performance in multi-document question answering (MD-QA)
  arXiv  Detail & Related papers  (2024-04-13T20:43:46Z)
• REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answering [115.72130322143275]
  REAR is a RElevance-Aware Retrieval-augmented approach for open-domain question answering (QA) We develop a novel architecture for LLM-based RAG systems, by incorporating a specially designed assessment module. Experiments on four open-domain QA tasks show that REAR significantly outperforms previous a number of competitive RAG approaches.
  arXiv  Detail & Related papers  (2024-02-27T13:22:51Z)
• Corrective Retrieval Augmented Generation [36.04062963574603]
  Retrieval-augmented generation (RAG) relies heavily on relevance of retrieved documents, raising concerns about how the model behaves if retrieval goes wrong. We propose the Corrective Retrieval Augmented Generation (CRAG) to improve the robustness of generation. CRAG is plug-and-play and can be seamlessly coupled with various RAG-based approaches.
  arXiv  Detail & Related papers  (2024-01-29T04:36:39Z)
• Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Context [4.1229332722825]
  This paper introduces a novel framework that combines graph-driven context retrieval in conjunction to knowledge graphs based enhancement. We conduct experiments on various Large Language Models (LLMs) with different parameter sizes to evaluate their ability to ground knowledge and determine factual accuracy in answers to open-ended questions. Our methodology GraphContextGen consistently outperforms dominant text-based retrieval systems, demonstrating its robustness and adaptability to a larger number of use cases.
  arXiv  Detail & Related papers  (2024-01-23T11:25:34Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Generator-Retriever-Generator Approach for Open-Domain Question Answering [18.950517545413813]
  We propose a novel approach that combines document retrieval techniques with a large language model (LLM) In parallel, a dual-encoder network retrieves documents that are relevant to the question from an external corpus. GRG outperforms the state-of-the-art generate-then-read and retrieve-then-read pipelines.
  arXiv  Detail & Related papers  (2023-07-21T00:34:38Z)
• Synergistic Interplay between Search and Large Language Models for Information Retrieval [141.18083677333848]
  InteR allows RMs to expand knowledge in queries using LLM-generated knowledge collections. InteR achieves overall superior zero-shot retrieval performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-12T11:58:15Z)
• Generate rather than Retrieve: Large Language Models are Strong Context Generators [74.87021992611672]
  We present a novel perspective for solving knowledge-intensive tasks by replacing document retrievers with large language model generators. We call our method generate-then-read (GenRead), which first prompts a large language model to generate contextutal documents based on a given question, and then reads the generated documents to produce the final answer.
  arXiv  Detail & Related papers  (2022-09-21T01:30:59Z)
• BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models [65.51390418485207]
  We propose a new approach of harvesting massive KGs of arbitrary relations from pretrained LMs. With minimal input of a relation definition, the approach efficiently searches in the vast entity pair space to extract diverse accurate knowledge. We deploy the approach to harvest KGs of over 400 new relations from different LMs.
  arXiv  Detail & Related papers  (2022-06-28T19:46:29Z)
• Improving Conversational Recommendation Systems' Quality with Context-Aware Item Meta Information [42.88448098873448]
  Conversational recommendation systems (CRS) engage with users by inferring user preferences from dialog history. Previous CRSs use knowledge graph (KG) based recommendation modules and integrate KG with language models for response generation. We propose a simple yet effective architecture comprising a pre-trained language model (PLM) and an item metadata encoder.
  arXiv  Detail & Related papers  (2021-12-15T14:12:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.