CLEVE: Contrastive Pre-training for Event Extraction

• URL: http://arxiv.org/abs/2105.14485v1
• Date: Sun, 30 May 2021 09:50:17 GMT
• Title: CLEVE: Contrastive Pre-training for Event Extraction
• Authors: Ziqi Wang, Xiaozhi Wang, Xu Han, Yankai Lin, Lei Hou, Zhiyuan Liu, Peng Li, Juanzi Li, Jie Zhou
• Abstract summary: CLEVE is a contrastive pre-training framework for event extraction. It learns event knowledge from large unsupervised data and semantic structures. Experiments on ACE 2005 and MAVEN datasets show that CLEVE achieves significant improvements.
• Score: 44.11464475981792
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event extraction (EE) has considerably benefited from pre-trained language models (PLMs) by fine-tuning. However, existing pre-training methods have not involved modeling event characteristics, resulting in the developed EE models cannot take full advantage of large-scale unsupervised data. To this end, we propose CLEVE, a contrastive pre-training framework for EE to better learn event knowledge from large unsupervised data and their semantic structures (e.g. AMR) obtained with automatic parsers. CLEVE contains a text encoder to learn event semantics and a graph encoder to learn event structures respectively. Specifically, the text encoder learns event semantic representations by self-supervised contrastive learning to represent the words of the same events closer than those unrelated words; the graph encoder learns event structure representations by graph contrastive pre-training on parsed event-related semantic structures. The two complementary representations then work together to improve both the conventional supervised EE and the unsupervised "liberal" EE, which requires jointly extracting events and discovering event schemata without any annotated data. Experiments on ACE 2005 and MAVEN datasets show that CLEVE achieves significant improvements, especially in the challenging unsupervised setting. The source code and pre-trained checkpoints can be obtained from https://github.com/THU-KEG/CLEVE.

Related papers

• What Would Happen Next? Predicting Consequences from An Event Causality Graph [23.92119748794742]
  Existing script event prediction task forcasts the subsequent event based on an event script chain. This paper introduces a Causality Graph Event Prediction task that forecasting consequential event based on an Event Causality Graph (ECG)
  arXiv  Detail & Related papers  (2024-09-26T02:34:08Z)
• DEGAP: Dual Event-Guided Adaptive Prefixes for Templated-Based Event Argument Extraction with Slot Querying [32.115904077731386]
  Recent advancements in event argument extraction (EAE) involve incorporating useful auxiliary information into models during training and inference. These methods face two challenges: (1) the retrieval results may be irrelevant and (2) templates are developed independently for each event without considering their possible relationship. We propose DEGAP to address these challenges through a simple yet effective components: dual prefixes, i.e. learnable prompt vectors, and an event-guided adaptive gating mechanism.
  arXiv  Detail & Related papers  (2024-05-22T03:56:55Z)
• GenEARL: A Training-Free Generative Framework for Multimodal Event Argument Role Labeling [89.07386210297373]
  GenEARL is a training-free generative framework that harnesses the power of modern generative models to understand event task descriptions. We show that GenEARL outperforms the contrastive pretraining (CLIP) baseline by 9.4% and 14.2% accuracy for zero-shot EARL on the M2E2 and SwiG datasets.
  arXiv  Detail & Related papers  (2024-04-07T00:28:13Z)
• Prompt-based Graph Model for Joint Liberal Event Extraction and Event Schema Induction [1.3154296174423619]
  Events are essential components of speech and texts, describing the changes in the state of entities. The event extraction task aims to identify and classify events and find their participants according to event schemas. The researchers propose Liberal Event Extraction (LEE), which aims to extract events and discover event schemas simultaneously.
  arXiv  Detail & Related papers  (2024-03-19T07:56:42Z)
• Semantic Structure Enhanced Event Causality Identification [57.26259734944247]
  Event Causality Identification (ECI) aims to identify causal relations between events in unstructured texts. Existing methods underestimate two kinds of semantic structures vital to the ECI task, namely, event-centric structure and event-associated structure.
  arXiv  Detail & Related papers  (2023-05-22T07:42:35Z)
• SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding [132.78015553111234]
  Hand gesture serves as a crucial role during the expression of sign language. Current deep learning based methods for sign language understanding (SLU) are prone to over-fitting due to insufficient sign data resource. We propose the first self-supervised pre-trainable SignBERT+ framework with model-aware hand prior incorporated.
  arXiv  Detail & Related papers  (2023-05-08T17:16:38Z)
• Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representations [70.41385310930846]
  We present an end-to-end framework Structure-CLIP to enhance multi-modal structured representations. We use scene graphs to guide the construction of semantic negative examples, which results in an increased emphasis on learning structured representations. A Knowledge-Enhance (KEE) is proposed to leverage SGK as input to further enhance structured representations.
  arXiv  Detail & Related papers  (2023-05-06T03:57:05Z)
• CLIP-Event: Connecting Text and Images with Event Structures [123.31452120399827]
  We propose a contrastive learning framework to enforce vision-language pretraining models. We take advantage of text information extraction technologies to obtain event structural knowledge. Experiments show that our zero-shot CLIP-Event outperforms the state-of-the-art supervised model in argument extraction.
  arXiv  Detail & Related papers  (2022-01-13T17:03:57Z)
• Text2Event: Controllable Sequence-to-Structure Generation for End-to-end Event Extraction [35.39643772926177]
  Event extraction is challenging due to the complex structure of event records and the semantic gap between text and event. Traditional methods usually extract event records by decomposing the complex structure prediction task into multiple subtasks. We propose Text2Event, a sequence-to-structure generation paradigm that can directly extract events from the text in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-06-17T04:00:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.