Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

• URL: http://arxiv.org/abs/2404.14741v3
• Date: Sun, 06 Oct 2024 10:55:23 GMT
• Title: Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering
• Authors: Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Kang Liu, Jun Zhao,
• Abstract summary: We propose a training-free method called Generate-on-Graph (GoG) to generate new factual triples while exploring Knowledge Graphs (KGs) GoG performs reasoning through a Thinking-Searching-Generating framework, which treats LLM as both Agent and KG in IKGQA.
• Score: 87.67177556994525
• License:
• Abstract: To address the issues of insufficient knowledge and hallucination in Large Language Models (LLMs), numerous studies have explored integrating LLMs with Knowledge Graphs (KGs). However, these methods are typically evaluated on conventional Knowledge Graph Question Answering (KGQA) with complete KGs, where all factual triples required for each question are entirely covered by the given KG. In such cases, LLMs primarily act as an agent to find answer entities within the KG, rather than effectively integrating the internal knowledge of LLMs and external knowledge sources such as KGs. In fact, KGs are often incomplete to cover all the knowledge required to answer questions. To simulate these real-world scenarios and evaluate the ability of LLMs to integrate internal and external knowledge, we propose leveraging LLMs for QA under Incomplete Knowledge Graph (IKGQA), where the provided KG lacks some of the factual triples for each question, and construct corresponding datasets. To handle IKGQA, we propose a training-free method called Generate-on-Graph (GoG), which can generate new factual triples while exploring KGs. Specifically, GoG performs reasoning through a Thinking-Searching-Generating framework, which treats LLM as both Agent and KG in IKGQA. Experimental results on two datasets demonstrate that our GoG outperforms all previous methods.

Related papers

• Decoding on Graphs: Faithful and Sound Reasoning on Knowledge Graphs through Generation of Well-Formed Chains [66.55612528039894]
  Knowledge Graphs (KGs) can serve as reliable knowledge sources for question answering (QA) We present DoG (Decoding on Graphs), a novel framework that facilitates a deep synergy between LLMs and KGs. Experiments across various KGQA tasks with different background KGs demonstrate that DoG achieves superior and robust performance.
  arXiv  Detail & Related papers  (2024-10-24T04:01:40Z)
• Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models [83.28737898989694]
  Large language models (LLMs) struggle with faithful reasoning due to knowledge gaps and hallucinations. We introduce graph-constrained reasoning (GCR), a novel framework that bridges structured knowledge in KGs with unstructured reasoning in LLMs. GCR achieves state-of-the-art performance and exhibits strong zero-shot generalizability to unseen KGs without additional training.
  arXiv  Detail & Related papers  (2024-10-16T22:55:17Z)
• A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning [17.676185326247946]
  We propose a prompt-based KG foundation model via in-context learning, namely KG-ICL, to achieve a universal reasoning ability. To encode prompt graphs with the generalization ability to unseen entities and relations in queries, we first propose a unified tokenizer. Then, we propose two message passing neural networks to perform prompt encoding and KG reasoning, respectively.
  arXiv  Detail & Related papers  (2024-10-16T06:47:18Z)
• Tree-of-Traversals: A Zero-Shot Reasoning Algorithm for Augmenting Black-box Language Models with Knowledge Graphs [72.89652710634051]
  Knowledge graphs (KGs) complement Large Language Models (LLMs) by providing reliable, structured, domain-specific, and up-to-date external knowledge. We introduce Tree-of-Traversals, a novel zero-shot reasoning algorithm that enables augmentation of black-box LLMs with one or more KGs.
  arXiv  Detail & Related papers  (2024-07-31T06:01:24Z)
• LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answering [1.5238808518078564]
  LinkQ is a system that leverages a large language model (LLM) to facilitate knowledge graph (KG) query construction through natural language question-answering. Our results indicate that practitioners find LinkQ effective for KG question-answering.
  arXiv  Detail & Related papers  (2024-06-07T15:28:31Z)
• KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph [134.8631016845467]
  We propose an autonomous LLM-based agent framework, called KG-Agent. In KG-Agent, we integrate the LLM, multifunctional toolbox, KG-based executor, and knowledge memory. To guarantee the effectiveness, we leverage program language to formulate the multi-hop reasoning process over the KG.
  arXiv  Detail & Related papers  (2024-02-17T02:07:49Z)
• Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering [16.434098552925427]
  We study the KG-augmented language model approach for solving the knowledge graph question answering (KGQA) task. We propose an answer-sensitive KG-to-Text approach that can transform KG knowledge into well-textualized statements.
  arXiv  Detail & Related papers  (2023-09-20T10:42:08Z)
• Unifying Large Language Models and Knowledge Graphs: A Roadmap [61.824618473293725]
  Large language models (LLMs) are making new waves in the field of natural language processing and artificial intelligence. Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge.
  arXiv  Detail & Related papers  (2023-06-14T07:15:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.