ForecastTKGQuestions: A Benchmark for Temporal Question Answering and Forecasting over Temporal Knowledge Graphs

• URL: http://arxiv.org/abs/2208.06501v2
• Date: Tue, 18 Jul 2023 15:05:49 GMT
• Title: ForecastTKGQuestions: A Benchmark for Temporal Question Answering and Forecasting over Temporal Knowledge Graphs
• Authors: Zifeng Ding, Zongyue Li, Ruoxia Qi, Jingpei Wu, Bailan He, Yunpu Ma, Zhao Meng, Shuo Chen, Ruotong Liao, Zhen Han, Volker Tresp
• Abstract summary: Question answering over temporal knowledge graphs (TKGQA) has recently found increasing interest. TKGQA requires temporal reasoning techniques to extract the relevant information from temporal knowledge bases. We propose a novel task: forecasting question answering over temporal knowledge graphs.
• Score: 28.434829347176233
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Question answering over temporal knowledge graphs (TKGQA) has recently found increasing interest. TKGQA requires temporal reasoning techniques to extract the relevant information from temporal knowledge bases. The only existing TKGQA dataset, i.e., CronQuestions, consists of temporal questions based on the facts from a fixed time period, where a temporal knowledge graph (TKG) spanning the same period can be fully used for answer inference, allowing the TKGQA models to use even the future knowledge to answer the questions based on the past facts. In real-world scenarios, however, it is also common that given the knowledge until now, we wish the TKGQA systems to answer the questions asking about the future. As humans constantly seek plans for the future, building TKGQA systems for answering such forecasting questions is important. Nevertheless, this has still been unexplored in previous research. In this paper, we propose a novel task: forecasting question answering over temporal knowledge graphs. We also propose a large-scale TKGQA benchmark dataset, i.e., ForecastTKGQuestions, for this task. It includes three types of questions, i.e., entity prediction, yes-no, and fact reasoning questions. For every forecasting question in our dataset, QA models can only have access to the TKG information before the timestamp annotated in the given question for answer inference. We find that the state-of-the-art TKGQA methods perform poorly on forecasting questions, and they are unable to answer yes-no questions and fact reasoning questions. To this end, we propose ForecastTKGQA, a TKGQA model that employs a TKG forecasting module for future inference, to answer all three types of questions. Experimental results show that ForecastTKGQA outperforms recent TKGQA methods on the entity prediction questions, and it also shows great effectiveness in answering the other two types of questions.

Related papers

• Temporal Knowledge Graph Question Answering: A Survey [39.40384139630724]
  Temporal Knowledge Graph Question Answering (TKGQA) is an emerging task to answer temporal questions. This paper provides a thorough survey from two perspectives: the taxonomy of temporal questions and the methodological categorization for TKGQA.
  arXiv  Detail & Related papers  (2024-06-20T10:51:06Z)
• Event Extraction as Question Generation and Answering [72.04433206754489]
  Recent work on Event Extraction has reframed the task as Question Answering (QA) We propose QGA-EE, which enables a Question Generation (QG) model to generate questions that incorporate rich contextual information instead of using fixed templates. Experiments show that QGA-EE outperforms all prior single-task-based models on the ACE05 English dataset.
  arXiv  Detail & Related papers  (2023-07-10T01:46:15Z)
• TwiRGCN: Temporally Weighted Graph Convolution for Question Answering over Temporal Knowledge Graphs [35.50055476282997]
  We show how to generalize relational graph convolutional networks (RGCN) for temporal question answering (QA) We propose a novel, intuitive and interpretable scheme to modulate the messages passed through a KG edge during convolution. We evaluate the resulting system, which we call TwiRGCN, on TimeQuestions, a recently released, challenging dataset for complex temporal QA.
  arXiv  Detail & Related papers  (2022-10-12T15:03:49Z)
• RealTime QA: What's the Answer Right Now? [137.04039209995932]
  We introduce REALTIME QA, a dynamic question answering (QA) platform that announces questions and evaluates systems on a regular basis. We build strong baseline models upon large pretrained language models, including GPT-3 and T5. GPT-3 tends to return outdated answers when retrieved documents do not provide sufficient information to find an answer.
  arXiv  Detail & Related papers  (2022-07-27T07:26:01Z)
• TempoQR: Temporal Question Reasoning over Knowledge Graphs [11.054877399064804]
  This paper puts forth a comprehensive embedding-based framework for answering complex questions over Knowledge Graphs. Our method termed temporal question reasoning (TempoQR) exploits TKG embeddings to ground the question to the specific entities and time scope it refers to. Experiments show that TempoQR improves accuracy by 25--45 percentage points on complex temporal questions over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-12-10T23:59:14Z)
• Relation-Guided Pre-Training for Open-Domain Question Answering [67.86958978322188]
  We propose a Relation-Guided Pre-Training (RGPT-QA) framework to solve complex open-domain questions. We show that RGPT-QA achieves 2.2%, 2.4%, and 6.3% absolute improvement in Exact Match accuracy on Natural Questions, TriviaQA, and WebQuestions.
  arXiv  Detail & Related papers  (2021-09-21T17:59:31Z)
• Complex Temporal Question Answering on Knowledge Graphs [22.996399822102575]
  This work presents EXAQT, the first end-to-end system for answering complex temporal questions. It answers natural language questions over knowledge graphs (KGs) in two stages, one geared towards high recall, the other towards precision at top ranks. We evaluate EXAQT on TimeQuestions, a large dataset of 16k temporal questions compiled from a variety of general purpose KG-QA benchmarks.
  arXiv  Detail & Related papers  (2021-09-18T13:41:43Z)
• Improving Unsupervised Question Answering via Summarization-Informed Question Generation [47.96911338198302]
  Question Generation (QG) is the task of generating a plausible question for a passage, answer> pair. We make use of freely available news summary data, transforming declarative sentences into appropriate questions using dependency parsing, named entity recognition and semantic role labeling. The resulting questions are then combined with the original news articles to train an end-to-end neural QG model.
  arXiv  Detail & Related papers  (2021-09-16T13:08:43Z)
• SituatedQA: Incorporating Extra-Linguistic Contexts into QA [7.495151447459443]
  We introduce SituatedQA, an open-retrieval QA dataset where systems must produce the correct answer to a question given the temporal or geographical context. We find that a significant proportion of information seeking questions have context-dependent answers. Our study shows that existing models struggle with producing answers that are frequently updated or from uncommon locations.
  arXiv  Detail & Related papers  (2021-09-13T17:53:21Z)
• A Dataset for Answering Time-Sensitive Questions [88.95075983560331]
  Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. It is important to consider the time dimension and empower the existing QA models to reason over time. The existing QA datasets contain rather few time-sensitive questions, hence not suitable for diagnosing or benchmarking the model's temporal reasoning capability.
  arXiv  Detail & Related papers  (2021-08-13T16:42:25Z)
• NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions [80.60423934589515]
  We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark. We set up multi-choice and open-ended QA tasks targeting causal action reasoning, temporal action reasoning, and common scene comprehension. We find that top-performing methods excel at shallow scene descriptions but are weak in causal and temporal action reasoning.
  arXiv  Detail & Related papers  (2021-05-18T04:56:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.