Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling

• URL: http://arxiv.org/abs/2306.11489v2
• Date: Tue, 30 Jan 2024 12:11:45 GMT
• Title: Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling
• Authors: Linyao Yang and Hongyang Chen and Zhao Li and Xiao Ding and Xindong Wu
• Abstract summary: ChatGPT, a representative large language model (LLM), has gained considerable attention due to its powerful emergent abilities. This paper proposes to enhance LLMs with knowledge graph-enhanced large language models (KGLLMs) KGLLM provides a solution to enhance LLMs' factual reasoning ability, opening up new avenues for LLM research.
• Score: 34.59678835272862
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, ChatGPT, a representative large language model (LLM), has gained considerable attention due to its powerful emergent abilities. Some researchers suggest that LLMs could potentially replace structured knowledge bases like knowledge graphs (KGs) and function as parameterized knowledge bases. However, while LLMs are proficient at learning probabilistic language patterns based on large corpus and engaging in conversations with humans, they, like previous smaller pre-trained language models (PLMs), still have difficulty in recalling facts while generating knowledge-grounded contents. To overcome these limitations, researchers have proposed enhancing data-driven PLMs with knowledge-based KGs to incorporate explicit factual knowledge into PLMs, thus improving their performance to generate texts requiring factual knowledge and providing more informed responses to user queries. This paper reviews the studies on enhancing PLMs with KGs, detailing existing knowledge graph enhanced pre-trained language models (KGPLMs) as well as their applications. Inspired by existing studies on KGPLM, this paper proposes to enhance LLMs with KGs by developing knowledge graph-enhanced large language models (KGLLMs). KGLLM provides a solution to enhance LLMs' factual reasoning ability, opening up new avenues for LLM research.

Related papers

• Large Language Models Can Better Understand Knowledge Graphs Than We Thought [13.336418752729987]
  knowledge graph (KG) embeddings with model parameters become increasingly costly. Current prompting methods often rely on a trial-and-error approach. We show that unordered linearized triples are more effective for LLMs' understanding of KGs compared to fluent NL text.
  arXiv  Detail & Related papers  (2024-02-18T10:44:03Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph [11.808990571175269]
  Large language model (LLM) has achieved outstanding performance on various downstream tasks with its powerful natural language understanding and zero-shot capability, but LLM still suffers from knowledge limitation. We propose a novel framework KnowledgeNavigator to address these challenges by efficiently and accurately retrieving external knowledge from knowledge graph. We evaluate KnowledgeNavigator on multiple public KGQA benchmarks, the experiments show the framework has great effectiveness and generalization.
  arXiv  Detail & Related papers  (2023-12-26T04:22:56Z)
• KnowGPT: Knowledge Graph based Prompting for Large Language Models [28.605161596626875]
  We introduce a Knowledge Graph based PrompTing framework, namely KnowGPT, to enhance Large Language Models with domain knowledge. KnowGPT contains a knowledge extraction module to extract the most informative knowledge from KGs, and a context-aware prompt construction module to automatically convert extracted knowledge into effective prompts. KnowGPT achieves a 92.6% accuracy on OpenbookQA leaderboard, comparable to human-level performance.
  arXiv  Detail & Related papers  (2023-12-11T07:56:25Z)
• Mitigating Large Language Model Hallucinations via Autonomous Knowledge Graph-based Retrofitting [51.7049140329611]
  This paper proposes Knowledge Graph-based Retrofitting (KGR) to mitigate factual hallucination during the reasoning process. Experiments show that KGR can significantly improve the performance of LLMs on factual QA benchmarks.
  arXiv  Detail & Related papers  (2023-11-22T11:08:38Z)
• Graph Neural Prompting with Large Language Models [32.97391910476073]
  Graph Neural Prompting (GNP) is a novel plug-and-play method to assist pre-trained language models in learning beneficial knowledge from knowledge graphs. Extensive experiments on multiple datasets demonstrate the superiority of GNP on both commonsense and biomedical reasoning tasks.
  arXiv  Detail & Related papers  (2023-09-27T06:33:29Z)
• Unifying Large Language Models and Knowledge Graphs: A Roadmap [61.824618473293725]
  Large language models (LLMs) are making new waves in the field of natural language processing and artificial intelligence. Knowledge Graphs (KGs), Wikipedia and Huapu for example, are structured knowledge models that explicitly store rich factual knowledge.
  arXiv  Detail & Related papers  (2023-06-14T07:15:26Z)
• Knowledge Rumination for Pre-trained Language Models [77.55888291165462]
  We propose a new paradigm dubbed Knowledge Rumination to help the pre-trained language model utilize related latent knowledge without retrieving it from the external corpus. We apply the proposed knowledge rumination to various language models, including RoBERTa, DeBERTa, and GPT-3.
  arXiv  Detail & Related papers  (2023-05-15T15:47:09Z)
• Augmented Large Language Models with Parametric Knowledge Guiding [72.71468058502228]
  Large Language Models (LLMs) have significantly advanced natural language processing (NLP) with their impressive language understanding and generation capabilities. Their performance may be suboptimal for domain-specific tasks that require specialized knowledge due to limited exposure to the related data. We propose the novel Parametric Knowledge Guiding (PKG) framework, which equips LLMs with a knowledge-guiding module to access relevant knowledge.
  arXiv  Detail & Related papers  (2023-05-08T15:05:16Z)
• Adapters for Enhanced Modeling of Multilingual Knowledge and Text [54.02078328453149]
  Language models have been extended to multilingual language models (MLLMs) Knowledge graphs contain facts in an explicit triple format, which require careful curation and are only available in a few high-resource languages. We propose to enhance MLLMs with knowledge from multilingual knowledge graphs (MLKGs) so as to tackle language and knowledge graph tasks across many languages.
  arXiv  Detail & Related papers  (2022-10-24T21:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.