MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation

• URL: http://arxiv.org/abs/2311.09105v2
• Date: Tue, 18 Jun 2024 22:15:39 GMT
• Title: MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation
• Authors: Xiaozhi Wang, Hao Peng, Yong Guan, Kaisheng Zeng, Jianhui Chen, Lei Hou, Xu Han, Yankai Lin, Zhiyuan Liu, Ruobing Xie, Jie Zhou, Juanzi Li,
• Abstract summary: MAVEN-Arg is the first all-in-one dataset supporting event detection, event argument extraction, and event relation extraction. As an EAE benchmark, MAVEN-Arg offers three main advantages: (1) a comprehensive schema covering 162 event types and 612 argument roles, all with expert-written definitions and examples; (2) a large data scale, containing 98,591 events and 290,613 arguments obtained with laborious human annotation; and (3) the exhaustive annotation supporting all task variants of EAE.
• Score: 104.6065882758648
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding events in texts is a core objective of natural language understanding, which requires detecting event occurrences, extracting event arguments, and analyzing inter-event relationships. However, due to the annotation challenges brought by task complexity, a large-scale dataset covering the full process of event understanding has long been absent. In this paper, we introduce MAVEN-Arg, which augments MAVEN datasets with event argument annotations, making the first all-in-one dataset supporting event detection, event argument extraction (EAE), and event relation extraction. As an EAE benchmark, MAVEN-Arg offers three main advantages: (1) a comprehensive schema covering 162 event types and 612 argument roles, all with expert-written definitions and examples; (2) a large data scale, containing 98,591 events and 290,613 arguments obtained with laborious human annotation; (3) the exhaustive annotation supporting all task variants of EAE, which annotates both entity and non-entity event arguments in document level. Experiments indicate that MAVEN-Arg is quite challenging for both fine-tuned EAE models and proprietary large language models (LLMs). Furthermore, to demonstrate the benefits of an all-in-one dataset, we preliminarily explore a potential application, future event prediction, with LLMs. MAVEN-Arg and codes can be obtained from https://github.com/THU-KEG/MAVEN-Argument.

Related papers

• MAVEN-Fact: A Large-scale Event Factuality Detection Dataset [55.01875707021496]
  We introduce MAVEN-Fact, a large-scale and high-quality EFD dataset based on the MAVEN dataset. MAVEN-Fact includes factuality annotations of 112,276 events, making it the largest EFD dataset. Experiments demonstrate that MAVEN-Fact is challenging for both conventional fine-tuned models and large language models (LLMs)
  arXiv  Detail & Related papers  (2024-07-22T03:43:46Z)
• From Simple to Complex: A Progressive Framework for Document-level Informative Argument Extraction [34.37013964529546]
  Event Argument Extraction (EAE) requires the model to extract arguments of multiple events from a single document. We propose a simple-to-complex progressive framework for document-level EAE. Our model outperforms SOTA by 1.4% in F1, indicating the proposed simple-to-complex framework is useful in the EAE task.
  arXiv  Detail & Related papers  (2023-10-25T04:38:02Z)
• MEE: A Novel Multilingual Event Extraction Dataset [62.80569691825534]
  Event Extraction aims to recognize event mentions and their arguments from text. The lack of high-quality multilingual EE datasets for model training and evaluation has been the main hindrance. We propose a novel Multilingual Event Extraction dataset (EE) that provides annotation for more than 50K event mentions in 8 typologically different languages.
  arXiv  Detail & Related papers  (2022-11-11T02:01:41Z)
• GENEVA: Benchmarking Generalizability for Event Argument Extraction with Hundreds of Event Types and Argument Roles [77.05288144035056]
  Event Argument Extraction (EAE) has focused on improving model generalizability to cater to new events and domains. Standard benchmarking datasets like ACE and ERE cover less than 40 event types and 25 entity-centric argument roles.
  arXiv  Detail & Related papers  (2022-05-25T05:46:28Z)
• Event Argument Extraction using Causal Knowledge Structures [9.56216681584111]
  Event Argument extraction refers to the task of extracting structured information from unstructured text for a particular event of interest. Most of the existing works model this task at a sentence level, restricting the context to a local scope. We propose an external knowledge aided approach to infuse document-level event information to aid the extraction of complex event arguments.
  arXiv  Detail & Related papers  (2021-05-02T13:59:07Z)
• ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning [49.795767003586235]
  We introduce ESTER, a comprehensive machine reading comprehension dataset for Event Semantic Relation Reasoning. We study five most commonly used event semantic relations and formulate them as question answering tasks. Experimental results show that the current SOTA systems achieve 60.5%, 57.8%, and 76.3% for event-based F1, token based F1 and HIT@1 scores respectively.
  arXiv  Detail & Related papers  (2021-04-16T19:59:26Z)
• Unsupervised Label-aware Event Trigger and Argument Classification [73.86358632937372]
  We propose an unsupervised event extraction pipeline, which first identifies events with available tools (e.g., SRL) and then automatically maps them to pre-defined event types. We leverage pre-trained language models to contextually represent pre-defined types for both event triggers and arguments. We successfully map 83% of the triggers and 54% of the arguments to the correct types, almost doubling the performance of previous zero-shot approaches.
  arXiv  Detail & Related papers  (2020-12-30T17:47:24Z)
• Document-level Event Extraction with Efficient End-to-end Learning of Cross-event Dependencies [37.96254956540803]
  We propose an end-to-end model leveraging Deep Value Networks (DVN), a structured prediction algorithm, to efficiently capture cross-event dependencies for document-level event extraction. Our approach achieves comparable performance to CRF-based models on ACE05, while enjoys significantly higher computational efficiency.
  arXiv  Detail & Related papers  (2020-10-24T05:28:16Z)
• Resource-Enhanced Neural Model for Event Argument Extraction [28.812507794694543]
  Event argument extraction aims to identify the arguments of an event and classify the roles that those arguments play. We propose a trigger-aware sequence encoder with several types of trigger-dependent sequence representations. Experiments on the English ACE2005 benchmark show that our approach achieves a new state-of-the-art.
  arXiv  Detail & Related papers  (2020-10-06T21:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.