Large Language Models Meet Knowledge Graphs to Answer Factoid Questions

• URL: http://arxiv.org/abs/2310.02166v1
• Date: Tue, 3 Oct 2023 15:57:00 GMT
• Title: Large Language Models Meet Knowledge Graphs to Answer Factoid Questions
• Authors: Mikhail Salnikov, Hai Le, Prateek Rajput, Irina Nikishina, Pavel Braslavski, Valentin Malykh and Alexander Panchenko
• Abstract summary: We propose a method for exploring pre-trained Text-to-Text Language Models enriched with additional information from Knowledge Graphs. We procure easily interpreted information with Transformer-based models through the linearization of the extracted subgraphs. Final re-ranking of the answer candidates with the extracted information boosts Hits@1 scores of the pre-trained text-to-text language models by 4-6%.
• Score: 57.47634017738877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, it has been shown that the incorporation of structured knowledge into Large Language Models significantly improves the results for a variety of NLP tasks. In this paper, we propose a method for exploring pre-trained Text-to-Text Language Models enriched with additional information from Knowledge Graphs for answering factoid questions. More specifically, we propose an algorithm for subgraphs extraction from a Knowledge Graph based on question entities and answer candidates. Then, we procure easily interpreted information with Transformer-based models through the linearization of the extracted subgraphs. Final re-ranking of the answer candidates with the extracted information boosts Hits@1 scores of the pre-trained text-to-text language models by 4-6%.

Related papers

• Question-guided Knowledge Graph Re-scoring and Injection for Knowledge Graph Question Answering [27.414670144354453]
  KGQA involves answering natural language questions by leveraging structured information stored in a knowledge graph. We propose a Question-guided Knowledge Graph Re-scoring method (Q-KGR) to eliminate noisy pathways for the input question. We also introduce Knowformer, a parameter-efficient method for injecting the re-scored knowledge graph into large language models to enhance their ability to perform factual reasoning.
  arXiv  Detail & Related papers  (2024-10-02T10:27:07Z)
• Infusing Knowledge into Large Language Models with Contextual Prompts [5.865016596356753]
  We propose a simple yet generalisable approach for knowledge infusion by generating prompts from the context in the input text. Our experiments show the effectiveness of our approach which we evaluate by probing the fine-tuned LLMs.
  arXiv  Detail & Related papers  (2024-03-03T11:19:26Z)
• Surveying the Landscape of Text Summarization with Deep Learning: A Comprehensive Review [2.4185510826808487]
  Deep learning has revolutionized natural language processing (NLP) by enabling the development of models that can learn complex representations of language data. Deep learning models for NLP typically use large amounts of data to train deep neural networks, allowing them to learn the patterns and relationships in language data. Applying deep learning to text summarization refers to the use of deep neural networks to perform text summarization tasks.
  arXiv  Detail & Related papers  (2023-10-13T21:24:37Z)
• Answer Candidate Type Selection: Text-to-Text Language Model for Closed Book Question Answering Meets Knowledge Graphs [62.20354845651949]
  We present a novel approach which works on top of the pre-trained Text-to-Text QA system to address this issue. Our simple yet effective method performs filtering and re-ranking of generated candidates based on their types derived from Wikidata "instance_of" property.
  arXiv  Detail & Related papers  (2023-10-10T20:49:43Z)
• ChatGraph: Interpretable Text Classification by Converting ChatGPT Knowledge to Graphs [54.48467003509595]
  ChatGPT has shown superior performance in various natural language processing (NLP) tasks. We propose a novel framework that leverages the power of ChatGPT for specific tasks, such as text classification. Our method provides a more transparent decision-making process compared with previous text classification methods.
  arXiv  Detail & Related papers  (2023-05-03T19:57:43Z)
• Retrieval augmentation of large language models for lay language generation [12.686922203465896]
  We introduce CELLS, the largest (63k pairs) and broadest-ranging (12 journals) parallel corpus for lay language generation. The abstract and the corresponding lay language summary are written by domain experts, assuring the quality of our dataset. We derive two specialized paired corpora from CELLS to address key challenges in lay language generation: generating background explanations and simplifying the original abstract.
  arXiv  Detail & Related papers  (2022-11-07T19:06:53Z)
• Deep Bidirectional Language-Knowledge Graph Pretraining [159.9645181522436]
  DRAGON is a self-supervised approach to pretraining a deeply joint language-knowledge foundation model from text and KG at scale. Our model takes pairs of text segments and relevant KG subgraphs as input and bidirectionally fuses information from both modalities.
  arXiv  Detail & Related papers  (2022-10-17T18:02:52Z)
• An Empirical Investigation of Commonsense Self-Supervision with Knowledge Graphs [67.23285413610243]
  Self-supervision based on the information extracted from large knowledge graphs has been shown to improve the generalization of language models. We study the effect of knowledge sampling strategies and sizes that can be used to generate synthetic data for adapting language models.
  arXiv  Detail & Related papers  (2022-05-21T19:49:04Z)
• Syntax-informed Question Answering with Heterogeneous Graph Transformer [2.139714421848487]
  We present a linguistics-informed question answering approach that extends and fine-tunes a pre-trained neural language model. We illustrate the approach by the addition of syntactic information in the form of dependency and constituency graphic structures connecting tokens and virtual tokens.
  arXiv  Detail & Related papers  (2022-04-01T07:48:03Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.