Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness

• URL: http://arxiv.org/abs/2004.08166v2
• Date: Sun, 14 Feb 2021 20:33:58 GMT
• Title: Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness
• Authors: Yavuz Selim Kartal, Busra Guvenen and Mucahid Kutlu
• Abstract summary: We propose a model prioritizing the claims based on their check-worthiness. We use BERT model with additional features including domain-specific controversial topics, word embeddings, and others.
• Score: 1.2891210250935146
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The massive amount of misinformation spreading on the Internet on a daily basis has enormous negative impacts on societies. Therefore, we need automated systems helping fact-checkers in the combat against misinformation. In this paper, we propose a model prioritizing the claims based on their check-worthiness. We use BERT model with additional features including domain-specific controversial topics, word embeddings, and others. In our experiments, we show that our proposed model outperforms all state-of-the-art models in both test collections of CLEF Check That! Lab in 2018 and 2019. We also conduct a qualitative analysis to shed light-detecting check-worthy claims. We suggest requesting rationales behind judgments are needed to understand subjective nature of the task and problematic labels.

Related papers

• From Chaos to Clarity: Claim Normalization to Empower Fact-Checking [57.024192702939736]
  Claim Normalization (aka ClaimNorm) aims to decompose complex and noisy social media posts into more straightforward and understandable forms. We propose CACN, a pioneering approach that leverages chain-of-thought and claim check-worthiness estimation. Our experiments demonstrate that CACN outperforms several baselines across various evaluation measures.
  arXiv  Detail & Related papers  (2023-10-22T16:07:06Z)
• Leveraging Social Discourse to Measure Check-worthiness of Claims for Fact-checking [36.21314290592325]
  We present CheckIt, a manually annotated large Twitter dataset for fine-grained claim check-worthiness. We benchmark our dataset against a unified approach, CheckMate, that jointly determines whether a claim is check-worthy and the factors that led to that conclusion.
  arXiv  Detail & Related papers  (2023-09-17T13:42:41Z)
• WiCE: Real-World Entailment for Claims in Wikipedia [63.234352061821625]
  We propose WiCE, a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia. In addition to standard claim-level entailment, WiCE provides entailment judgments over sub-sentence units of the claim. We show that real claims in our dataset involve challenging verification and retrieval problems that existing models fail to address.
  arXiv  Detail & Related papers  (2023-03-02T17:45:32Z)
• Check-worthy Claim Detection across Topics for Automated Fact-checking [21.723689314962233]
  We assess and quantify the challenge of detecting check-worthy claims for new, unseen topics. We propose the AraCWA model to mitigate the performance deterioration when detecting check-worthy claims across topics.
  arXiv  Detail & Related papers  (2022-12-16T14:54:56Z)
• Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twitter [25.944789217337338]
  Claim Span Identification (CSI) is a tool to automatically identify and extract the snippets of claim-worthy (mis)information present in a post. We propose CURT, a large-scale Twitter corpus with token-level claim spans on more than 7.5k tweets. We benchmark our dataset with DABERTa, an adapter-based variation of RoBERTa.
  arXiv  Detail & Related papers  (2022-10-10T14:08:46Z)
• Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems [80.3811072650087]
  We show that it is possible to subtly modify claim-salient snippets in the evidence and generate diverse and claim-aligned evidence. The attacks are also robust against post-hoc modifications of the claim. These attacks can have harmful implications on the inspectable and human-in-the-loop usage scenarios.
  arXiv  Detail & Related papers  (2022-09-07T13:39:24Z)
• Synthetic Disinformation Attacks on Automated Fact Verification Systems [53.011635547834025]
  We explore the sensitivity of automated fact-checkers to synthetic adversarial evidence in two simulated settings. We show that these systems suffer significant performance drops against these attacks. We discuss the growing threat of modern NLG systems as generators of disinformation.
  arXiv  Detail & Related papers  (2022-02-18T19:01:01Z)
• Attacking Open-domain Question Answering by Injecting Misinformation [116.25434773461465]
  We study the risk of misinformation to Question Answering (QA) models by investigating the sensitivity of open-domain QA models to misinformation documents. Experiments show that QA models are vulnerable to even small amounts of evidence contamination brought by misinformation. We discuss the necessity of building a misinformation-aware QA system that integrates question-answering and misinformation detection.
  arXiv  Detail & Related papers  (2021-10-15T01:55:18Z)
• Leveraging Commonsense Knowledge on Classifying False News and Determining Checkworthiness of Claims [1.487444917213389]
  We propose to leverage commonsense knowledge for the tasks of false news classification and check-worthy claim detection. We fine-tune the BERT language model with a commonsense question answering task and the aforementioned tasks in a multi-task learning environment. Our experimental analysis demonstrates that commonsense knowledge can improve performance in both tasks.
  arXiv  Detail & Related papers  (2021-08-08T20:52:45Z)
• FaVIQ: FAct Verification from Information-seeking Questions [77.7067957445298]
  We construct a large-scale fact verification dataset called FaVIQ using information-seeking questions posed by real users. Our claims are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.
  arXiv  Detail & Related papers  (2021-07-05T17:31:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.