UCD-CS at W-NUT 2020 Shared Task-3: A Text to Text Approach for COVID-19 Event Extraction on Social Media

• URL: http://arxiv.org/abs/2009.10047v2
• Date: Mon, 12 Oct 2020 16:18:53 GMT
• Title: UCD-CS at W-NUT 2020 Shared Task-3: A Text to Text Approach for COVID-19 Event Extraction on Social Media
• Abstract summary: We describe our approach in the shared task: COVID-19 event extraction from Twitter. The objective of this task is to extract answers from COVID-related tweets to a set of predefined slot-filling questions. Our approach treats the event extraction task as a question answering task by leveraging the transformer-based T5 text-to-text model.
• Score: 3.5585697639325113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we describe our approach in the shared task: COVID-19 event extraction from Twitter. The objective of this task is to extract answers from COVID-related tweets to a set of predefined slot-filling questions. Our approach treats the event extraction task as a question answering task by leveraging the transformer-based T5 text-to-text model. According to the official evaluation scores returned, namely F1, our submitted run achieves competitive performance compared to other participating runs (Top 3). However, we argue that this evaluation may underestimate the actual performance of runs based on text-generation. Although some such runs may answer the slot questions well, they may not be an exact string match for the gold standard answers. To measure the extent of this underestimation, we adopt a simple exact-answer transformation method aiming at converting the well-answered predictions to exactly-matched predictions. The results show that after this transformation our run overall reaches the same level of performance as the best participating run and state-of-the-art F1 scores in three of five COVID-related events. Our code is publicly available to aid reproducibility

Related papers

• DS@GT at CheckThat! 2025: Evaluating Context and Tokenization Strategies for Numerical Fact Verification [49.1574468325115]
  Numerical claims, statements involving quantities, comparisons, and temporal references pose unique challenges for automated fact-checking systems.<n>We evaluate modeling strategies for veracity prediction of such claims using the QuanTemp dataset and building our own evidence retrieval pipeline.<n>Our best-performing system achieves competitive macro-average F1 score of 0.57 and places us among the Top-4 submissions in Task 3 of CheckThat! 2025.
  arXiv  Detail & Related papers  (2025-07-08T17:22:22Z)
• Preference-based Learning with Retrieval Augmented Generation for Conversational Question Answering [20.969921246457414]
  PRAISE is a pipeline-based approach for ConvQA that trains adapters for each of the three subtasks. We show that PRAISE shows improvements per subtask and achieves new state-of-the-art performance on a popular ConvQA benchmark.
  arXiv  Detail & Related papers  (2025-03-28T10:26:49Z)
• An Energy-based Model for Word-level AutoCompletion in Computer-aided Translation [97.3797716862478]
  Word-level AutoCompletion (WLAC) is a rewarding yet challenging task in Computer-aided Translation. Existing work addresses this task through a classification model based on a neural network that maps the hidden vector of the input context into its corresponding label. This work proposes an energy-based model for WLAC, which enables the context hidden vector to capture crucial information from the source sentence.
  arXiv  Detail & Related papers  (2024-07-29T15:07:19Z)
• Towards Event Extraction from Speech with Contextual Clues [61.164413398231254]
  We introduce the Speech Event Extraction (SpeechEE) task and construct three synthetic training sets and one human-spoken test set. Compared to event extraction from text, SpeechEE poses greater challenges mainly due to complex speech signals that are continuous and have no word boundaries. Our method brings significant improvements on all datasets, achieving a maximum F1 gain of 10.7%.
  arXiv  Detail & Related papers  (2024-01-27T11:07:19Z)
• Link Prediction for Wikipedia Articles as a Natural Language Inference Task [1.1842520528140819]
  This paper introduces an approach to link prediction in Wikipedia articles by formulating it as a natural language inference (NLI) task. We implement our system based on the Sentence Pair Classification for Link Prediction for the Wikipedia Articles task. Our system achieved 0.99996 Macro F1-score and 1.00000 Macro F1-score for the public and private test sets, respectively.
  arXiv  Detail & Related papers  (2023-08-31T05:25:04Z)
• ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images [198.35937007558078]
  The competition opened on 30th December, 2022 and closed on 24th March, 2023. There are 35 participants and 91 valid submissions received for Track 1, and 15 participants and 26 valid submissions received for Track 2. According to the performance of the submissions, we believe there is still a large gap on the expected information extraction performance for complex and zero-shot scenarios.
  arXiv  Detail & Related papers  (2023-06-05T22:20:52Z)
• MIA 2022 Shared Task Submission: Leveraging Entity Representations, Dense-Sparse Hybrids, and Fusion-in-Decoder for Cross-Lingual Question Answering [5.837881923712393]
  We describe our two-stage system for the Multilingual Information Access (MIA) 2022 Shared Task on Cross-Lingual Open-Retrieval Question Answering. The first stage consists of multilingual passage retrieval with a hybrid dense and sparse retrieval strategy. The second stage consists of a reader which outputs the answer from the top passages returned by the first stage.
  arXiv  Detail & Related papers  (2022-07-05T10:27:17Z)
• UniTE: Unified Translation Evaluation [63.58868113074476]
  UniTE is the first unified framework engaged with abilities to handle all three evaluation tasks. We testify our framework on WMT 2019 Metrics and WMT 2020 Quality Estimation benchmarks.
  arXiv  Detail & Related papers  (2022-04-28T08:35:26Z)
• Zero-Shot Information Extraction as a Unified Text-to-Triple Translation [56.01830747416606]
  We cast a suite of information extraction tasks into a text-to-triple translation framework. We formalize the task as a translation between task-specific input text and output triples. We study the zero-shot performance of this framework on open information extraction.
  arXiv  Detail & Related papers  (2021-09-23T06:54:19Z)
• Exploiting Unsupervised Data for Emotion Recognition in Conversations [76.01690906995286]
  Emotion Recognition in Conversations (ERC) aims to predict the emotional state of speakers in conversations. The available supervised data for the ERC task is limited. We propose a novel approach to leverage unsupervised conversation data.
  arXiv  Detail & Related papers  (2020-10-02T13:28:47Z)
• Phonemer at WNUT-2020 Task 2: Sequence Classification Using COVID Twitter BERT and Bagging Ensemble Technique based on Plurality Voting [0.0]
  We develop a system that automatically identifies whether an English Tweet related to the novel coronavirus (COVID-19) is informative or not. Our final approach achieved an F1-score of 0.9037 and we were ranked sixth overall with F1-score as the evaluation criteria.
  arXiv  Detail & Related papers  (2020-10-01T10:54:54Z)
• TEST_POSITIVE at W-NUT 2020 Shared Task-3: Joint Event Multi-task Learning for Slot Filling in Noisy Text [26.270447944466557]
  We propose the Joint Event Multi-task Learning (JOELIN) model for extracting COVID-19 events from Twitter. Through a unified global learning framework, we make use of all the training data across different events to learn and fine-tune the language model. We implement a type-aware post-processing procedure using named entity recognition (NER) to further filter the predictions.
  arXiv  Detail & Related papers  (2020-09-29T19:08:45Z)
• BUT-FIT at SemEval-2020 Task 5: Automatic detection of counterfactual statements with deep pre-trained language representation models [6.853018135783218]
  This paper describes BUT-FIT's submission at SemEval-2020 Task 5: Modelling Causal Reasoning in Language: Detecting Counterfactuals. The challenge focused on detecting whether a given statement contains a counterfactual. We found RoBERTa LRM to perform the best in both subtasks.
  arXiv  Detail & Related papers  (2020-07-28T11:16:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.