Fraunhofer SIT at CheckThat! 2023: Tackling Classification Uncertainty Using Model Souping on the Example of Check-Worthiness Classification

• URL: http://arxiv.org/abs/2307.02377v2
• Date: Thu, 27 Jul 2023 14:43:56 GMT
• Title: Fraunhofer SIT at CheckThat! 2023: Tackling Classification Uncertainty Using Model Souping on the Example of Check-Worthiness Classification
• Authors: Raphael Frick, Inna Vogel, and Jeong-Eun Choi
• Abstract summary: This paper describes the second-placed approach developed by the Fraunhofer SIT team in the CLEF-2023 CheckThat! lab Task 1B for English. Given a text snippet from a political debate, the aim of this task is to determine whether it should be assessed for check-worthiness.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper describes the second-placed approach developed by the Fraunhofer SIT team in the CLEF-2023 CheckThat! lab Task 1B for English. Given a text snippet from a political debate, the aim of this task is to determine whether it should be assessed for check-worthiness. Detecting check-worthy statements aims to facilitate manual fact-checking efforts by prioritizing the claims that fact-checkers should consider first. It can also be considered as primary step of a fact-checking system. Our best-performing method took advantage of an ensemble classification scheme centered on Model Souping. When applied to the English data set, our submitted model achieved an overall F1 score of 0.878 and was ranked as the second-best model in the competition.

Related papers

• Ta'keed: The First Generative Fact-Checking System for Arabic Claims [0.0]
  This paper introduces Ta'keed, an explainable Arabic automatic fact-checking system. Ta'keed generates explanations for claim credibility, particularly in Arabic. The system achieved a promising F1 score of 0.72 in the classification task.
  arXiv  Detail & Related papers  (2024-01-25T10:43:00Z)
• JutePestDetect: An Intelligent Approach for Jute Pest Identification Using Fine-Tuned Transfer Learning [0.0]
  In certain Asian countries, Jute is prone to pest infestations, and its identification is typically made visually in countries like Bangladesh, India, Myanmar, and China. This study proposes a high-performing and resilient transfer learning (TL) based JutePestDetect model to identify jute pests at the early stage.
  arXiv  Detail & Related papers  (2023-05-28T15:51:35Z)
• Controlling Formality in Low-Resource NMT with Domain Adaptation and Re-Ranking: SLT-CDT-UoS at IWSLT2022 [4.348327991071386]
  This paper describes the SLT-CDT-UoS group's submission to the first Special Task on Formality Control for Spoken Language Translation. Our efforts were split between two fronts: data engineering and altering the objective function for best hypothesis selection. On the test sets for English-to-German and English-to-Spanish, we achieved an average accuracy of.935 within the constrained setting and.995 within unconstrained setting.
  arXiv  Detail & Related papers  (2022-05-12T09:54:17Z)
• Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees [63.62448343531963]
  We propose a combination of the existing paradigms, sampling responses to be scored by humans intelligently. We observe significant gains in accuracy (19.80% increase on average) and quadratic weighted kappa (QWK) (25.60% on average) with a relatively small human budget.
  arXiv  Detail & Related papers  (2021-11-17T05:00:51Z)
• Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims [26.96108180116284]
  CheckThat! lab featured two tasks in two different languages: English and Arabic. The most successful approaches to Task 1 used various neural networks and logistic regression. Learning-to-rank was used by the highest scoring runs for subtask A.
  arXiv  Detail & Related papers  (2021-09-25T16:08:09Z)
• Two-Stream Consensus Network: Submission to HACS Challenge 2021 Weakly-Supervised Learning Track [78.64815984927425]
  The goal of weakly-supervised temporal action localization is to temporally locate and classify action of interest in untrimmed videos. We adopt the two-stream consensus network (TSCN) as the main framework in this challenge. Our solution ranked 2rd in this challenge, and we hope our method can serve as a baseline for future academic research.
  arXiv  Detail & Related papers  (2021-06-21T03:36:36Z)
• Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection [85.53263670166304]
  One-stage detector basically formulates object detection as dense classification and localization. Recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization.
  arXiv  Detail & Related papers  (2020-06-08T07:24:33Z)
• L2R2: Leveraging Ranking for Abductive Reasoning [65.40375542988416]
  The abductive natural language inference task ($alpha$NLI) is proposed to evaluate the abductive reasoning ability of a learning system. A novel $L2R2$ approach is proposed under the learning-to-rank framework. Experiments on the ART dataset reach the state-of-the-art in the public leaderboard.
  arXiv  Detail & Related papers  (2020-05-22T15:01:23Z)
• Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning [61.32992639292889]
  Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. We introduce a new scoring method that casts a plausibility ranking task in a full-text format. We show that our method provides a much more stable training phase across random restarts.
  arXiv  Detail & Related papers  (2020-04-29T10:54:40Z)
• Claim Check-Worthiness Detection as Positive Unlabelled Learning [53.24606510691877]
  Claim check-worthiness detection is a critical component of fact checking systems. We illuminate a central challenge in claim check-worthiness detection underlying all of these tasks. Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning.
  arXiv  Detail & Related papers  (2020-03-05T16:06:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.