MarkQA: A large scale KBQA dataset with numerical reasoning

• URL: http://arxiv.org/abs/2310.15517v2
• Date: Thu, 14 Dec 2023 04:06:36 GMT
• Title: MarkQA: A large scale KBQA dataset with numerical reasoning
• Authors: Xiang Huang, Sitao Cheng, Yuheng Bao, Shanshan Huang, Yuzhong Qu
• Abstract summary: We propose a new task, NR-KBQA, which requires the ability to perform both multi-hop reasoning and numerical reasoning. We design a logic form in Python format called PyQL to represent the reasoning process of numerical reasoning questions. We present a large dataset called MarkQA, which is automatically constructed from a small set of seeds.
• Score: 11.072552105311484
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While question answering over knowledge bases (KBQA) has shown progress in addressing factoid questions, KBQA with numerical reasoning remains relatively unexplored. In this paper, we focus on the complex numerical reasoning in KBQA and propose a new task, NR-KBQA, which necessitates the ability to perform both multi-hop reasoning and numerical reasoning. We design a logic form in Python format called PyQL to represent the reasoning process of numerical reasoning questions. To facilitate the development of NR-KBQA, we present a large dataset called MarkQA, which is automatically constructed from a small set of seeds. Each question in MarkQA is equipped with its corresponding SPARQL query, alongside the step-by-step reasoning process in the QDMR format and PyQL program. Experimental results of some state-of-the-art QA methods on the MarkQA show that complex numerical reasoning in KBQA faces great challenges.

Related papers

• Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation [19.327008532572645]
  Question Generation over Knowledge Bases (KBQG) aims to convert a logical form into a natural language question. We propose Chain-of-Thought prompting, which is an in-context learning strategy for reasoning. We conduct extensive experiments over three public KBQG datasets.
  arXiv  Detail & Related papers  (2023-10-12T15:08:14Z)
• RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering [87.18962441714976]
  We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA) We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging. Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.
  arXiv  Detail & Related papers  (2022-10-25T21:39:36Z)
• Knowledge Base Question Answering by Case-based Reasoning over Subgraphs [81.22050011503933]
  We show that our model answers queries requiring complex reasoning patterns more effectively than existing KG completion algorithms. The proposed model outperforms or performs competitively with state-of-the-art models on several KBQA benchmarks.
  arXiv  Detail & Related papers  (2022-02-22T01:34:35Z)
• QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation [54.136509061542775]
  Multi-hop question generation (MQG) aims to generate complex questions which require reasoning over multiple pieces of information of the input passage. We propose a novel framework, QA4QG, a QA-augmented BART-based framework for MQG. Our results on the HotpotQA dataset show that QA4QG outperforms all state-of-the-art models.
  arXiv  Detail & Related papers  (2022-02-14T08:16:47Z)
• A Benchmark for Generalizable and Interpretable Temporal Question Answering over Knowledge Bases [67.33560134350427]
  TempQA-WD is a benchmark dataset for temporal reasoning. It is based on Wikidata, which is the most frequently curated, openly available knowledge base.
  arXiv  Detail & Related papers  (2022-01-15T08:49:09Z)
• Efficient Contextualization using Top-k Operators for Question Answering over Knowledge Graphs [24.520002698010856]
  This work presents ECQA, an efficient method that prunes irrelevant parts of the search space using KB-aware signals. Experiments with two recent QA benchmarks demonstrate the superiority of ECQA over state-of-the-art baselines with respect to answer presence, size of the search space, and runtimes.
  arXiv  Detail & Related papers  (2021-08-19T10:06:14Z)
• A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges [71.4531144086568]
  Question Answering (QA) over Knowledge Base (KB) aims to automatically answer natural language questions. Researchers have shifted their attention from simple questions to complex questions, which require more KB triples and constraint inference.
  arXiv  Detail & Related papers  (2020-07-26T07:13:32Z)
• KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base [67.87878113432723]
  We introduce KQA Pro, a dataset for Complex KBQA including 120K diverse natural language questions. For each question, we provide the corresponding KoPL program and SPARQL query, so that KQA Pro serves for both KBQA and semantic parsing tasks.
  arXiv  Detail & Related papers  (2020-07-08T03:28:04Z)
• A Complex KBQA System using Multiple Reasoning Paths [42.007327947635595]
  Multi-hop knowledge based question answering (KBQA) is a complex task for natural language understanding. We introduce an end-to-end KBQA system which can leverage multiple reasoning paths' information and only requires labeled answer as supervision.
  arXiv  Detail & Related papers  (2020-05-22T02:35:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.