Check-worthy Claim Detection across Topics for Automated Fact-checking

• URL: http://arxiv.org/abs/2212.08514v1
• Date: Fri, 16 Dec 2022 14:54:56 GMT
• Title: Check-worthy Claim Detection across Topics for Automated Fact-checking
• Authors: Amani S. Abumansour, Arkaitz Zubiaga
• Abstract summary: We assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. We propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics.
• Score: 21.723689314962233
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An important component of an automated fact-checking system is the claim check-worthiness detection system, which ranks sentences by prioritising them based on their need to be checked. Despite a body of research tackling the task, previous research has overlooked the challenging nature of identifying check-worthy claims across different topics. In this paper, we assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. After highlighting the problem, we propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics. The AraCWA model enables boosting the performance for new topics by incorporating two components for few-shot learning and data augmentation. Using a publicly available dataset of Arabic tweets consisting of 14 different topics, we demonstrate that our proposed data augmentation strategy achieves substantial improvements across topics overall, where the extent of the improvement varies across topics. Further, we analyse the semantic similarities between topics, suggesting that the similarity metric could be used as a proxy to determine the difficulty level of an unseen topic prior to undertaking the task of labelling the underlying sentences.

Related papers

• Supporting Automated Fact-checking across Topics: Similarity-driven Gradual Topic Learning for Claim Detection [10.80303881266859]
  We propose a domain-adaptation framework for check-worthy claims detection across topics for the Arabic language. We propose the Gradual Topic Learning (GTL) model, which builds an ability to learning gradually and emphasizes the check-worthy claims for the target topic. In addition, we introduce the Similarity-driven Gradual Topic Learning (SGTL) model that synthesizes gradual learning with a similarity-based strategy for the target topic.
  arXiv  Detail & Related papers  (2024-11-08T10:24:00Z)
• Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detection [82.65760006883248]
  We introduce a new task named Change Detection Question Answering and Grounding (CDQAG) CDQAG extends the traditional change detection task by providing interpretable textual answers and intuitive visual evidence. We construct the first CDQAG benchmark dataset, termed QAG-360K, comprising over 360K triplets of questions, textual answers, and corresponding high-quality visual masks.
  arXiv  Detail & Related papers  (2024-10-31T11:20:13Z)
• IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection [1.3686993145787067]
  This paper describes IAI group's participation for automated check-worthiness estimation for claims. The task involves the automated detection of check-worthy claims in English, Dutch, and Arabic political debates and Twitter data. We utilize various pre-trained generative decoder and encoder transformer models, employing methods such as few-shot chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2024-08-02T08:59:09Z)
• Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research [7.242609314791262]
  We present state-of-the-art multilingual claim detection research categorized into three key factors of the problem, verifiability, priority, and similarity. We present a detailed overview of the existing multilingual datasets along with the challenges and suggest possible future advancements.
  arXiv  Detail & Related papers  (2024-01-22T14:17:03Z)
• MythQA: Query-Based Large-Scale Check-Worthy Claim Detection through Multi-Answer Open-Domain Question Answering [8.70509665552136]
  Check-worthy claim detection aims at providing plausible misinformation to downstream fact-checking systems or human experts to check. Many efforts have been put into how to identify check-worthy claims from a small scale of pre-collected claims, but how to efficiently detect check-worthy claims directly from a large-scale information source, such as Twitter, remains underexplored. We introduce MythQA, a new multi-answer open-domain question answering(QA) task that involves contradictory stance mining for query-based large-scale check-worthy claim detection.
  arXiv  Detail & Related papers  (2023-07-21T18:35:24Z)
• ReAct: Temporal Action Detection with Relational Queries [84.76646044604055]
  This work aims at advancing temporal action detection (TAD) using an encoder-decoder framework with action queries. We first propose a relational attention mechanism in the decoder, which guides the attention among queries based on their relations. Lastly, we propose to predict the localization quality of each action query at inference in order to distinguish high-quality queries.
  arXiv  Detail & Related papers  (2022-07-14T17:46:37Z)
• Exposing Query Identification for Search Transparency [69.06545074617685]
  We explore the feasibility of approximate exposing query identification (EQI) as a retrieval task by reversing the role of queries and documents in two classes of search systems. We derive an evaluation metric to measure the quality of a ranking of exposing queries, as well as conducting an empirical analysis focusing on various practical aspects of approximate EQI.
  arXiv  Detail & Related papers  (2021-10-14T20:19:27Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Unsupervised Domain Adaption of Object Detectors: A Survey [87.08473838767235]
  Recent advances in deep learning have led to the development of accurate and efficient models for various computer vision applications. Learning highly accurate models relies on the availability of datasets with a large number of annotated images. Due to this, model performance drops drastically when evaluated on label-scarce datasets having visually distinct images.
  arXiv  Detail & Related papers  (2021-05-27T23:34:06Z)
• Analysing the Effect of Clarifying Questions on Document Ranking in Conversational Search [10.335808358080289]
  We investigate how different aspects of clarifying questions and user answers affect the quality of ranking. We introduce a simple-based lexical baseline, that significantly outperforms the existing naive baselines.
  arXiv  Detail & Related papers  (2020-08-09T12:55:16Z)
• Claim Check-Worthiness Detection as Positive Unlabelled Learning [53.24606510691877]
  Claim check-worthiness detection is a critical component of fact checking systems. We illuminate a central challenge in claim check-worthiness detection underlying all of these tasks. Our best performing method is a unified approach which automatically corrects for this using a variant of positive unlabelled learning.
  arXiv  Detail & Related papers  (2020-03-05T16:06:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.