Once Upon a $\textit{Time}$ in $\textit{Graph}$: Relative-Time Pretraining for Complex Temporal Reasoning

• URL: http://arxiv.org/abs/2310.14709v1
• Date: Mon, 23 Oct 2023 08:49:00 GMT
• Title: Once Upon a $\textit{Time}$ in $\textit{Graph}$: Relative-Time Pretraining for Complex Temporal Reasoning
• Authors: Sen Yang, Xin Li, Lidong Bing, Wai Lam
• Abstract summary: We make use of the underlying nature of time, and suggest creating a graph structure based on the relative placements of events along the time axis. Inspired by the graph view, we propose RemeMo, which explicitly connects all temporally-scoped facts by modeling the time relations between any two sentences. Experimental results show that RemeMo outperforms the baseline T5 on multiple temporal question answering datasets.
• Score: 96.03608822291136
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Our physical world is constantly evolving over time, rendering challenges for pre-trained language models to understand and reason over the temporal contexts of texts. Existing work focuses on strengthening the direct association between a piece of text and its time-stamp. However, the knowledge-time association is usually insufficient for the downstream tasks that require reasoning over temporal dependencies between knowledge. In this work, we make use of the underlying nature of time, all temporally-scoped sentences are strung together through a one-dimensional time axis, and suggest creating a graph structure based on the relative placements of events along the time axis. Inspired by the graph view, we propose RemeMo ($\underline{Re}$lative Ti$\underline{me}$ $\underline{Mo}$deling), which explicitly connects all temporally-scoped facts by modeling the time relations between any two sentences. Experimental results show that RemeMo outperforms the baseline T5 on multiple temporal question answering datasets under various settings. Further analysis suggests that RemeMo is especially good at modeling long-range complex temporal dependencies. We release our code and pre-trained checkpoints at $\href{https://github.com/DAMO-NLP-SG/RemeMo}{\text{this url}}$.

Related papers

• Joint Multi-Facts Reasoning Network For Complex Temporal Question Answering Over Knowledge Graph [34.44840297353777]
  Temporal Knowledge Graph (TKG) is an extension of regular knowledge graph by attaching the time scope. We propose textbfunderlineJoint textbfunderlineMulti textbfunderlineFacts textbfunderlineReasoning textbfunderlineNetwork (JMFRN)
  arXiv  Detail & Related papers  (2024-01-04T11:34:39Z)
• MTGER: Multi-view Temporal Graph Enhanced Temporal Reasoning over Time-Involved Document [26.26604509399347]
  MTGER is a novel framework for temporal reasoning over time-involved documents. It explicitly models the temporal relationships among facts by multi-view temporal graphs. We show that MTGER gives more consistent answers under question perturbations.
  arXiv  Detail & Related papers  (2023-11-08T16:41:37Z)
• More than Classification: A Unified Framework for Event Temporal Relation Extraction [61.44799147458621]
  Event temporal relation extraction(ETRE) is usually formulated as a multi-label classification task. We observe that all relations can be interpreted using the start and end time points of events. We propose a unified event temporal relation extraction framework, which transforms temporal relations into logical expressions of time points.
  arXiv  Detail & Related papers  (2023-05-28T02:09:08Z)
• Scanning Only Once: An End-to-end Framework for Fast Temporal Grounding in Long Videos [60.86880787242561]
  Video temporal grounding aims to pinpoint a video segment that matches the query description. We propose an end-to-end framework for fast temporal grounding, which is able to model an hours-long video with textbfone-time network execution. Our method significantly outperforms state-of-the-arts, and achieves textbf14.6$times$ / textbf102.8$times$ higher efficiency respectively.
  arXiv  Detail & Related papers  (2023-03-15T03:54:43Z)
• HiSMatch: Historical Structure Matching based Temporal Knowledge Graph Reasoning [59.38797474903334]
  This paper proposes the textbfHistorical textbfStructure textbfMatching (textbfHiSMatch) model. It applies two structure encoders to capture the semantic information contained in the historical structures of the query and candidate entities. Experiments on six benchmark datasets demonstrate the significant improvement of the proposed HiSMatch model, with up to 5.6% performance improvement in MRR, compared to the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-10-18T09:39:26Z)
• A Dataset for Answering Time-Sensitive Questions [88.95075983560331]
  Time is an important dimension in our physical world. Lots of facts can evolve with respect to time. It is important to consider the time dimension and empower the existing QA models to reason over time. The existing QA datasets contain rather few time-sensitive questions, hence not suitable for diagnosing or benchmarking the model's temporal reasoning capability.
  arXiv  Detail & Related papers  (2021-08-13T16:42:25Z)
• TIMEDIAL: Temporal Commonsense Reasoning in Dialog [43.24596551545824]
  We present the first study to investigate pre-trained language models for their temporal reasoning capabilities in dialogs. We formulate TIME-DIAL as a multiple-choice cloze task with over 1.1K carefully curated dialogs. Empirical results demonstrate that even the best performing models struggle on this task compared to humans.
  arXiv  Detail & Related papers  (2021-06-08T17:59:21Z)
• Exploring Contextualized Neural Language Models for Temporal Dependency Parsing [10.17066263304299]
  We show that BERT significantly improves temporal dependency parsing. We also present a detailed analysis on why deep contextualized neural LMs help and where they may fall short.
  arXiv  Detail & Related papers  (2020-04-30T03:59:13Z)
• Local-Global Video-Text Interactions for Temporal Grounding [77.5114709695216]
  This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query. We tackle this problem using a novel regression-based model that learns to extract a collection of mid-level features for semantic phrases in a text query. The proposed method effectively predicts the target time interval by exploiting contextual information from local to global.
  arXiv  Detail & Related papers  (2020-04-16T08:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.