ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification

• URL: http://arxiv.org/abs/2203.02225v1
• Date: Fri, 4 Mar 2022 10:11:15 GMT
• Title: ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification
• Authors: Yucheng Zhou, Tao Shen, Xiubo Geng, Guodong Long, Daxin Jiang
• Abstract summary: We propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning. The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios.
• Score: 74.6318379374801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating new events given context with correlated ones plays a crucial role in many event-centric reasoning tasks. Existing works either limit their scope to specific scenarios or overlook event-level correlations. In this paper, we propose to pre-train a general Correlation-aware context-to-Event Transformer (ClarET) for event-centric reasoning. To achieve this, we propose three novel event-centric objectives, i.e., whole event recovering, contrastive event-correlation encoding and prompt-based event locating, which highlight event-level correlations with effective training. The proposed ClarET is applicable to a wide range of event-centric reasoning scenarios, considering its versatility of (i) event-correlation types (e.g., causal, temporal, contrast), (ii) application formulations (i.e., generation and classification), and (iii) reasoning types (e.g., abductive, counterfactual and ending reasoning). Empirical fine-tuning results, as well as zero- and few-shot learning, on 9 benchmarks (5 generation and 4 classification tasks covering 4 reasoning types with diverse event correlations), verify its effectiveness and generalization ability.

Related papers

• EVIT: Event-Oriented Instruction Tuning for Event Reasoning [18.012724531672813]
  Event reasoning aims to infer events according to certain relations and predict future events. Large language models (LLMs) have made significant advancements in event reasoning owing to their wealth of knowledge and reasoning capabilities. However, smaller instruction-tuned models currently in use do not consistently demonstrate exceptional proficiency in managing these tasks.
  arXiv  Detail & Related papers  (2024-04-18T08:14:53Z)
• Enhancing Event Causality Identification with Rationale and Structure-Aware Causal Question Answering [30.000134835133522]
  Event Causality Identification (DECI) aims to identify causal relations between two events in documents. Recent research tends to use pre-trained language models to generate the event causal relations. We propose a multi-task learning framework to enhance event causality identification with rationale and structure-aware causal question answering.
  arXiv  Detail & Related papers  (2024-03-17T07:41:58Z)
• Improving Event Definition Following For Zero-Shot Event Detection [66.27883872707523]
  Existing approaches on zero-shot event detection usually train models on datasets annotated with known event types. We aim to improve zero-shot event detection by training models to better follow event definitions.
  arXiv  Detail & Related papers  (2024-03-05T01:46:50Z)
• Event Causality Extraction with Event Argument Correlations [13.403222002600558]
  Event Causality Extraction aims to extract cause-effect event causality pairs from plain texts. We propose a method with a dual grid tagging scheme to capture the intra- and inter-event argument correlations for ECE.
  arXiv  Detail & Related papers  (2023-01-27T09:48:31Z)
• Towards Out-of-Distribution Sequential Event Prediction: A Causal Treatment [72.50906475214457]
  The goal of sequential event prediction is to estimate the next event based on a sequence of historical events. In practice, the next-event prediction models are trained with sequential data collected at one time. We propose a framework with hierarchical branching structures for learning context-specific representations.
  arXiv  Detail & Related papers  (2022-10-24T07:54:13Z)
• Event-Centric Question Answering via Contrastive Learning and Invertible Event Transformation [29.60817278635999]
  We propose a novel QA model with contrastive learning and invertible event transformation, called TranCLR. Our proposed model utilizes an invertible transformation matrix to project semantic vectors of events into a common event embedding space, trained with contrastive learning, and thus naturally inject event semantic knowledge into mainstream QA pipelines.
  arXiv  Detail & Related papers  (2022-10-24T01:15:06Z)
• EA$^2$E: Improving Consistency with Event Awareness for Document-Level Argument Extraction [52.43978926985928]
  We introduce the Event-Aware Argument Extraction (EA$2$E) model with augmented context for training and inference. Experiment results on WIKIEVENTS and ACE2005 datasets demonstrate the effectiveness of EA$2$E.
  arXiv  Detail & Related papers  (2022-05-30T04:33:51Z)
• Learning Constraints and Descriptive Segmentation for Subevent Detection [74.48201657623218]
  We propose an approach to learning and enforcing constraints that capture dependencies between subevent detection and EventSeg prediction. We adopt Rectifier Networks for constraint learning and then convert the learned constraints to a regularization term in the loss function of the neural model.
  arXiv  Detail & Related papers  (2021-09-13T20:50:37Z)
• ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning [49.795767003586235]
  We introduce ESTER, a comprehensive machine reading comprehension dataset for Event Semantic Relation Reasoning. We study five most commonly used event semantic relations and formulate them as question answering tasks. Experimental results show that the current SOTA systems achieve 60.5%, 57.8%, and 76.3% for event-based F1, token based F1 and HIT@1 scores respectively.
  arXiv  Detail & Related papers  (2021-04-16T19:59:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.