EVENT5Ws: A Large Dataset for Open-Domain Event Extraction from Documents

• URL: http://arxiv.org/abs/2604.21890v1
• Date: Thu, 23 Apr 2026 17:42:07 GMT
• Title: EVENT5Ws: A Large Dataset for Open-Domain Event Extraction from Documents
• Abstract summary: EVENT5Ws is a large, manually annotated, and statistically verified open-domain event extraction dataset.<n>We evaluate state-of-the-art pre-trained large language models and establish a benchmark for future research.
• Score: 0.15749416770494704
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event extraction identifies the central aspects of events from text. It supports event understanding and analysis, which is crucial for tasks such as informed decision-making in emergencies. Therefore, it is necessary to develop automated event extraction approaches. However, existing datasets for algorithm development have limitations, including limited coverage of event types in closed-domain settings and a lack of large, manually verified dataset in open-domain settings. To address these limitations, we create EVENT5Ws , a large, manually annotated, and statistically verified open-domain event extraction dataset. We design a systematic annotation pipeline to create the dataset and provide empirical insights into annotation complexity. Using EVENT5Ws, we evaluate state-of-the-art pre-trained large language models and establish a benchmark for future research. We further show that models trained on EVENT5Ws generalize effectively to datasets from different geographical contexts, which demonstrates its potential for developing generalizable algorithms. Finally, we summarize the lessons learned during the dataset development and provide recommendations to support future large-scale dataset development.

Related papers

• SpatialBench: Is Your Spatial Foundation Model an All-Round Player? [92.031716744172]
  SpatialBench is a cross-paradigm, domain-diverse benchmark for spatial foundation models with deterministic sampling.<n>It comprehensively evaluates 41 models across 6 paradigms on 5 task suites under 4 different input density settings.<n>Our evaluations in challenging embodied and egocentric tasks demonstrate that strict domain alignment and high data quality are far more critical to performance than simple dataset scaling.
  arXiv  Detail & Related papers  (2026-05-26T17:59:20Z)
• A Multimodal Text- and Graph-Based Approach for Open-Domain Event Extraction from Documents [0.0]
  We present MODEE, a novel approach for open-domain event extraction.<n>It combines graph-based learning with text-based representation from LLMs to model document-level reasoning.<n> Empirical evaluations on large datasets demonstrate that MODEE outperforms state-of-the-art open-domain event extraction approaches.
  arXiv  Detail & Related papers  (2026-04-23T17:33:44Z)
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• Forecasting Future International Events: A Reliable Dataset for Text-Based Event Modeling [37.508538729757404]
  WorldREP is a novel dataset designed to address limitations by leveraging the advanced reasoning capabilities of large-language models (LLMs) Our dataset features high-quality scoring labels generated through advanced prompt modeling and rigorously validated by domain experts in political science. We publicly release our dataset along with the full automation source code for data collection, labeling, and benchmarking, aiming to support and advance research in text-based event prediction.
  arXiv  Detail & Related papers  (2024-11-21T11:44:23Z)
• Enabling Advanced Land Cover Analytics: An Integrated Data Extraction Pipeline for Predictive Modeling with the Dynamic World Dataset [1.3757956340051605]
  We present a flexible and efficient end to end pipeline for working with the Dynamic World dataset.<n>This includes a pre-processing and representation framework which tackles noise removal, efficient extraction of large amounts of data, and re-representation of LULC data.<n>To demonstrate the power of our pipeline, we use it to extract data for an urbanization prediction problem and build a suite of machine learning models with excellent performance.
  arXiv  Detail & Related papers  (2024-10-11T16:13:01Z)
• Capture the Flag: Uncovering Data Insights with Large Language Models [90.47038584812925]
  This study explores the potential of using Large Language Models (LLMs) to automate the discovery of insights in data. We propose a new evaluation methodology based on a "capture the flag" principle, measuring the ability of such models to recognize meaningful and pertinent information (flags) in a dataset.
  arXiv  Detail & Related papers  (2023-12-21T14:20:06Z)
• SCTc-TE: A Comprehensive Formulation and Benchmark for Temporal Event Forecasting [63.01035584154509]
  We develop a fully automated pipeline and construct a large-scale dataset named MidEast-TE from about 0.6 million news articles. This dataset focuses on the cooperation and conflict events among countries mainly in the MidEast region from 2015 to 2022. We propose a novel method LoGo that is able to take advantage of both Local and Global contexts for SCTc-TE forecasting.
  arXiv  Detail & Related papers  (2023-12-02T07:40:21Z)
• LargeST: A Benchmark Dataset for Large-Scale Traffic Forecasting [65.71129509623587]
  Road traffic forecasting plays a critical role in smart city initiatives and has experienced significant advancements thanks to the power of deep learning. However, the promising results achieved on current public datasets may not be applicable to practical scenarios. We introduce the LargeST benchmark dataset, which includes a total of 8,600 sensors in California with a 5-year time coverage.
  arXiv  Detail & Related papers  (2023-06-14T05:48:36Z)
• Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning [10.57079240576682]
  We introduce a novel Open-Set Self-Supervised Learning problem under the assumption that a large-scale unlabeled open-set is available. In our problem setup, it is crucial to consider the distribution mismatch between the open-set and target dataset. We demonstrate that SimCore significantly improves representation learning performance through extensive experimental settings.
  arXiv  Detail & Related papers  (2023-03-20T13:38:29Z)
• Challenges for Open-domain Targeted Sentiment Analysis [21.61943346030794]
  We propose a novel dataset consisting of 6,013 human-labeled data to extend the data domains in topics of interest and document level. We also offer a nested target annotation schema to extract the complete sentiment information in documents.
  arXiv  Detail & Related papers  (2022-04-14T11:44:02Z)
• Towards Inheritable Models for Open-Set Domain Adaptation [56.930641754944915]
  We introduce a practical Domain Adaptation paradigm where a source-trained model is used to facilitate adaptation in the absence of the source dataset in future. We present an objective way to quantify inheritability to enable the selection of the most suitable source model for a given target domain, even in the absence of the source data.
  arXiv  Detail & Related papers  (2020-04-09T07:16:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.