Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events

• URL: http://arxiv.org/abs/2004.04295v1
• Date: Wed, 8 Apr 2020 23:17:10 GMT
• Title: Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events
• Authors: Miguel Ballesteros, Rishita Anubhai, Shuai Wang, Nima Pourdamghani, Yogarshi Vyas, Jie Ma, Parminder Bhatia, Kathleen McKeown, and Yaser Al-Onaizan
• Abstract summary: We propose a neural architecture and a set of training methods for ordering events by predicting temporal relations. Given that a key challenge with this task is the scarcity of annotated data, our models rely on either pretrained representations or transfer and multi-task learning. Experiments on the MATRES dataset of English documents establish a new state-of-the-art on this task.
• Score: 41.35277143634441
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a neural architecture and a set of training methods for ordering events by predicting temporal relations. Our proposed models receive a pair of events within a span of text as input and they identify temporal relations (Before, After, Equal, Vague) between them. Given that a key challenge with this task is the scarcity of annotated data, our models rely on either pretrained representations (i.e. RoBERTa, BERT or ELMo), transfer and multi-task learning (by leveraging complementary datasets), and self-training techniques. Experiments on the MATRES dataset of English documents establish a new state-of-the-art on this task.

Related papers

• Large Language Models as Event Forecasters [10.32127659470566]
  Key elements of human events are extracted as quadruples that consist of subject, relation, object, and timestamp. These quadruples or quintuples, when organized within a specific domain, form a temporal knowledge graph (TKG)
  arXiv  Detail & Related papers  (2024-06-15T04:09:31Z)
• Learning to Rank Context for Named Entity Recognition Using a Synthetic Dataset [6.633914491587503]
  We propose to generate a synthetic context retrieval training dataset using Alpaca. Using this dataset, we train a neural context retriever based on a BERT model that is able to find relevant context for NER. We show that our method outperforms several retrieval baselines for the NER task on an English literary dataset composed of the first chapter of 40 books.
  arXiv  Detail & Related papers  (2023-10-16T06:53:12Z)
• Multimodal Inverse Cloze Task for Knowledge-based Visual Question Answering [4.114444605090133]
  We present a new pre-training method, Multimodal Inverse Cloze Task, for Knowledge-based Visual Question Answering about named Entities. KVQAE is a recently introduced task that consists in answering questions about named entities grounded in a visual context using a Knowledge Base. Our method is applicable to different neural network architectures and leads to a 9% relative-MRR and 15% relative-F1 gain for retrieval and reading comprehension.
  arXiv  Detail & Related papers  (2023-01-11T09:16:34Z)
• HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training [49.52679453475878]
  We propose a Temporal-Aware video-language pre-training framework, HiTeA, for modeling cross-modal alignment between moments and texts. We achieve state-of-the-art results on 15 well-established video-language understanding and generation tasks.
  arXiv  Detail & Related papers  (2022-12-30T04:27:01Z)
• Curriculum-Based Self-Training Makes Better Few-Shot Learners for Data-to-Text Generation [56.98033565736974]
  We propose Curriculum-Based Self-Training (CBST) to leverage unlabeled data in a rearranged order determined by the difficulty of text generation. Our method can outperform fine-tuning and task-adaptive pre-training methods, and achieve state-of-the-art performance in the few-shot setting of data-to-text generation.
  arXiv  Detail & Related papers  (2022-06-06T16:11:58Z)
• BiTimeBERT: Extending Pre-Trained Language Representations with Bi-Temporal Information [41.683057041628125]
  We introduce BiTimeBERT, a novel language representation model trained on a temporal collection of news articles. The experimental results show that BiTimeBERT consistently outperforms BERT and other existing pre-trained models.
  arXiv  Detail & Related papers  (2022-04-27T16:20:09Z)
• Improving Multi-task Generalization Ability for Neural Text Matching via Prompt Learning [54.66399120084227]
  Recent state-of-the-art neural text matching models (PLMs) are hard to generalize to different tasks. We adopt a specialization-generalization training strategy and refer to it as Match-Prompt. In specialization stage, descriptions of different matching tasks are mapped to only a few prompt tokens. In generalization stage, text matching model explores the essential matching signals by being trained on diverse multiple matching tasks.
  arXiv  Detail & Related papers  (2022-04-06T11:01:08Z)
• Syntax-Enhanced Pre-trained Model [49.1659635460369]
  We study the problem of leveraging the syntactic structure of text to enhance pre-trained models such as BERT and RoBERTa. Existing methods utilize syntax of text either in the pre-training stage or in the fine-tuning stage, so that they suffer from discrepancy between the two stages. We present a model that utilizes the syntax of text in both pre-training and fine-tuning stages.
  arXiv  Detail & Related papers  (2020-12-28T06:48:04Z)
• Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
  We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting. Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking. We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
  arXiv  Detail & Related papers  (2020-10-24T11:55:28Z)
• Modeling Preconditions in Text with a Crowd-sourced Dataset [17.828175478279654]
  This paper introduces PeKo, a crowd-sourced annotation of preconditions between event pairs in newswire. We also introduce two challenge tasks aimed at modeling preconditions. Evaluation on both tasks shows that modeling preconditions is challenging even for today's large language models.
  arXiv  Detail & Related papers  (2020-10-06T01:52:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.