Leveraging LLMs in Scholarly Knowledge Graph Question Answering

• URL: http://arxiv.org/abs/2311.09841v1
• Date: Thu, 16 Nov 2023 12:13:49 GMT
• Title: Leveraging LLMs in Scholarly Knowledge Graph Question Answering
• Authors: Tilahun Abedissa Taffa and Ricardo Usbeck
• Abstract summary: KGQA answers natural language questions by leveraging a large language model (LLM) Our system achieves an F1 score of 99.0% on SciQA - one of the Scholarly Knowledge Graph Question Answering challenge benchmarks.
• Score: 7.951847862547378
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a scholarly Knowledge Graph Question Answering (KGQA) that answers bibliographic natural language questions by leveraging a large language model (LLM) in a few-shot manner. The model initially identifies the top-n similar training questions related to a given test question via a BERT-based sentence encoder and retrieves their corresponding SPARQL. Using the top-n similar question-SPARQL pairs as an example and the test question creates a prompt. Then pass the prompt to the LLM and generate a SPARQL. Finally, runs the SPARQL against the underlying KG - ORKG (Open Research KG) endpoint and returns an answer. Our system achieves an F1 score of 99.0%, on SciQA - one of the Scholarly-QALD-23 challenge benchmarks.

Related papers

• Integrating SPARQL and LLMs for Question Answering over Scholarly Data Sources [0.0]
  This paper describes a methodology that combines SPARQL queries, divide and conquer algorithms, and BERT-based-case-SQuad2 predictions. The approach, evaluated with Exact Match and F-score metrics, shows promise for improving QA accuracy and efficiency in scholarly contexts.
  arXiv  Detail & Related papers  (2024-09-11T14:50:28Z)
• S-EQA: Tackling Situational Queries in Embodied Question Answering [48.43453390717167]
  We present and tackle the problem of Embodied Question Answering with Situational Queries (S-EQA) in a household environment. We first introduce a novel Prompt-Generate-Evaluate scheme that wraps around an LLM's output to create a dataset of unique situational queries and corresponding consensus object information. We report an improved accuracy of 15.31% while using queries framed from the generated object consensus for Visual Question Answering (VQA) over directly answering situational ones.
  arXiv  Detail & Related papers  (2024-05-08T00:45:20Z)
• In-Context Learning for Knowledge Base Question Answering for Unmanned Systems based on Large Language Models [43.642717344626355]
  We focus on the CCKS2023 Competition of Question Answering with Knowledge Graph Inference for Unmanned Systems. Inspired by the recent success of large language models (LLMs) like ChatGPT and GPT-3 in many QA tasks, we propose a ChatGPT-based Cypher Query Language (CQL) generation framework. With our ChatGPT-based CQL generation framework, we achieved the second place in the CCKS 2023 Question Answering with Knowledge Graph Inference for Unmanned Systems competition.
  arXiv  Detail & Related papers  (2023-11-06T08:52:11Z)
• An In-Context Schema Understanding Method for Knowledge Base Question Answering [70.87993081445127]
  Large Language Models (LLMs) have shown strong capabilities in language understanding and can be used to solve this task. Existing methods bypass this challenge by initially employing LLMs to generate drafts of logic forms without schema-specific details. We propose a simple In-Context Understanding (ICSU) method that enables LLMs to directly understand schemas by leveraging in-context learning.
  arXiv  Detail & Related papers  (2023-10-22T04:19:17Z)
• Open-Set Knowledge-Based Visual Question Answering with Inference Paths [79.55742631375063]
  The purpose of Knowledge-Based Visual Question Answering (KB-VQA) is to provide a correct answer to the question with the aid of external knowledge bases. We propose a new retriever-ranker paradigm of KB-VQA, Graph pATH rankER (GATHER for brevity) Specifically, it contains graph constructing, pruning, and path-level ranking, which not only retrieves accurate answers but also provides inference paths that explain the reasoning process.
  arXiv  Detail & Related papers  (2023-10-12T09:12:50Z)
• Spider4SPARQL: A Complex Benchmark for Evaluating Knowledge Graph Question Answering Systems [1.4732811715354452]
  It has become increasingly important to provide realistic benchmarks for evaluating Knowledge Graph Question Answering systems. Spider4SPARQL is a new SPARQL benchmark dataset featuring 9,693 previously existing manually generated NL questions and 4,721 unique, novel, and complex SPARQL queries. We evaluate the system with state-of-the-art KGQA systems as well as LLMs, which achieve only up to 45% execution accuracy.
  arXiv  Detail & Related papers  (2023-09-28T08:41:08Z)
• AutoQGS: Auto-Prompt for Low-Resource Knowledge-based Question Generation from SPARQL [18.019353543946913]
  This study investigates the task of knowledge-based question generation (KBQG) Conventional KBQG works generated questions from fact triples in the knowledge graph, which could not express complex operations like aggregation and comparison in SPARQL. We propose an auto-prompter trained on large-scale unsupervised data to rephrase SPARQL into NL description.
  arXiv  Detail & Related papers  (2022-08-26T06:53:46Z)
• DUAL: Textless Spoken Question Answering with Speech Discrete Unit Adaptive Learning [66.71308154398176]
  Spoken Question Answering (SQA) has gained research attention and made remarkable progress in recent years. Existing SQA methods rely on Automatic Speech Recognition (ASR) transcripts, which are time and cost-prohibitive to collect. This work proposes an ASR transcript-free SQA framework named Discrete Unit Adaptive Learning (DUAL), which leverages unlabeled data for pre-training and is fine-tuned by the SQA downstream task.
  arXiv  Detail & Related papers  (2022-03-09T17:46:22Z)
• SPBERT: Pre-training BERT on SPARQL Queries for End-to-end Question Answering over Knowledge Graphs [1.1775939485654976]
  SPBERT is a Transformer-based language model pre-trained on massive SPARQL query logs. We investigate how SPBERT and encoder-decoder architecture can be adapted for Knowledge-based QA corpora.
  arXiv  Detail & Related papers  (2021-06-18T08:39:26Z)
• Open Question Answering over Tables and Text [55.8412170633547]
  In open question answering (QA), the answer to a question is produced by retrieving and then analyzing documents that might contain answers to the question. Most open QA systems have considered only retrieving information from unstructured text. We present a new large-scale dataset Open Table-and-Text Question Answering (OTT-QA) to evaluate performance on this task.
  arXiv  Detail & Related papers  (2020-10-20T16:48:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.