CuriousLLM: Elevating Multi-Document QA with Reasoning-Infused Knowledge Graph Prompting

• URL: http://arxiv.org/abs/2404.09077v1
• Date: Sat, 13 Apr 2024 20:43:46 GMT
• Title: CuriousLLM: Elevating Multi-Document QA with Reasoning-Infused Knowledge Graph Prompting
• Authors: Zukang Yang, Zixuan Zhu,
• Abstract summary: We improve over a novel approach called Knowledge Graph Prompting (KGP), which combines knowledge graphs with a LLM-based agent to improve reasoning and search accuracy. We propose a reasoning-infused LLM agent to enhance this framework. This agent mimics human curiosity to ask follow-up questions to more efficiently navigate the search.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the field of Question Answering (QA), unifying large language models (LLMs) with external databases has shown great success. However, these methods often fall short in providing the advanced reasoning needed for complex QA tasks. To address these issues, we improve over a novel approach called Knowledge Graph Prompting (KGP), which combines knowledge graphs with a LLM-based agent to improve reasoning and search accuracy. Nevertheless, the original KGP framework necessitates costly fine-tuning with large datasets yet still suffers from LLM hallucination. Therefore, we propose a reasoning-infused LLM agent to enhance this framework. This agent mimics human curiosity to ask follow-up questions to more efficiently navigate the search. This simple modification significantly boosts the LLM performance in QA tasks without the high costs and latency associated with the initial KGP framework. Our ultimate goal is to further develop this approach, leading to more accurate, faster, and cost-effective solutions in the QA domain.

Related papers

• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation [81.18701211912779]
  We introduce an Adaptive Multi-Aspect Retrieval-augmented over KGs (Amar) framework. This method retrieves knowledge including entities, relations, and subgraphs, and converts each piece of retrieved text into prompt embeddings. Our method has achieved state-of-the-art performance on two common datasets.
  arXiv  Detail & Related papers  (2024-12-24T16:38:04Z)
• HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs [9.559336828884808]
  Large Language Models (LLMs) are adept at answering simple (single-hop) questions. As the complexity of the questions increase, the performance of LLMs degrades. Recent methods try to reduce this burden by integrating structured knowledge triples into the raw text. We propose to use a knowledge graph (KG) that is context-aware and is distilled to contain query-relevant information.
  arXiv  Detail & Related papers  (2024-06-10T05:22:49Z)
• Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts [50.06633829833144]
  Large Language Models (LLMs) are effective in performing various NLP tasks, but struggle to handle tasks that require extensive, real-world knowledge. We propose a benchmark that requires knowledge of long-tail facts for answering the involved questions. Our experiments show that LLMs alone struggle with answering these questions, especially when the long-tail level is high or rich knowledge is required.
  arXiv  Detail & Related papers  (2024-05-10T15:10:20Z)
• LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generation [6.676337039829463]
  We propose a Missing Information Guided Retrieve-Extraction-Solving paradigm (MIGRES) We leverage the identification of missing information to generate a targeted query that steers the subsequent knowledge retrieval. Extensive experiments conducted on multiple public datasets reveal the superiority of the proposed MIGRES method.
  arXiv  Detail & Related papers  (2024-04-22T09:56:59Z)
• Automatic Question-Answer Generation for Long-Tail Knowledge [65.11554185687258]
  We propose an automatic approach to generate specialized QA datasets for tail entities. We conduct extensive experiments by employing pretrained LLMs on our newly generated long-tail QA datasets.
  arXiv  Detail & Related papers  (2024-03-03T03:06:31Z)
• SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph [0.0]
  We evaluate strategies for fine-tuning the OpenLlama LLM for question answering over life science knowledge graphs. We propose an end-to-end data augmentation approach for extending a set of existing queries over a given knowledge graph. We also investigate the role of semantic "clues" in the queries, such as meaningful variable names and inline comments.
  arXiv  Detail & Related papers  (2024-02-07T07:24:01Z)
• Context Matters: Pushing the Boundaries of Open-Ended Answer Generation with Graph-Structured Knowledge Context [4.1229332722825]
  This paper introduces a novel framework that combines graph-driven context retrieval in conjunction to knowledge graphs based enhancement. We conduct experiments on various Large Language Models (LLMs) with different parameter sizes to evaluate their ability to ground knowledge and determine factual accuracy in answers to open-ended questions. Our methodology GraphContextGen consistently outperforms dominant text-based retrieval systems, demonstrating its robustness and adaptability to a larger number of use cases.
  arXiv  Detail & Related papers  (2024-01-23T11:25:34Z)
• keqing: knowledge-based question answering is a nature chain-of-thought mentor of LLM [27.76205400533089]
  Large language models (LLMs) have exhibited remarkable performance on various natural language processing (NLP) tasks, especially for question answering. We present a novel framework to assist LLMs, such as ChatGPT, to retrieve question-related structured information on the knowledge graph. The experimental results on KBQA datasets show that Keqing can achieve competitive performance and illustrate the logic of answering each question.
  arXiv  Detail & Related papers  (2023-12-31T08:39:04Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation [109.8527403904657]
  We show that large language models (LLMs) possess unwavering confidence in their knowledge and cannot handle the conflict between internal and external knowledge well. Retrieval augmentation proves to be an effective approach in enhancing LLMs' awareness of knowledge boundaries. We propose a simple method to dynamically utilize supporting documents with our judgement strategy.
  arXiv  Detail & Related papers  (2023-07-20T16:46:10Z)
• Search-in-the-Chain: Interactively Enhancing Large Language Models with Search for Knowledge-intensive Tasks [121.74957524305283]
  This paper proposes a novel framework named textbfSearch-in-the-Chain (SearChain) for the interaction between Information Retrieval (IR) and Large Language Model (LLM) Experiments show that SearChain outperforms state-of-the-art baselines on complex knowledge-intensive tasks.
  arXiv  Detail & Related papers  (2023-04-28T10:15:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.