What do people want to fact-check?

• URL: http://arxiv.org/abs/2602.10935v1
• Date: Wed, 11 Feb 2026 15:14:54 GMT
• Title: What do people want to fact-check?
• Authors: Bijean Ghafouri, Dorsaf Sallami, Luca Luceri, Taylor Lynn Curtis, Jean-Francois Godbout, Emilio Ferrara, Reihaneh Rabbany,
• Abstract summary: We show that when people can fact-check anything they want, what do they actually ask about?<n>We analyze statements submitted by 457 participants to an open-ended AI fact-checking system.<n>First, users range widely across topics but default to a narrow epistemic repertoire, overwhelmingly submitting simple descriptive claims about present-day observables.<n>Second, roughly one in four requests concerns statements that cannot be empirically resolved, including moral judgments, speculative predictions, and subjective evaluations.<n>Third, comparison with the FEVER benchmark dataset exposes sharp structural divergences across all five dimensions, indicating that standard evaluation corpora encodes a synthetic claim environment
• Score: 10.451649710564775
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Research on misinformation has focused almost exclusively on supply, asking what falsehoods circulate, who produces them, and whether corrections work. A basic demand-side question remains unanswered. When ordinary people can fact-check anything they want, what do they actually ask about? We provide the first large-scale evidence on this question by analyzing close to 2{,}500 statements submitted by 457 participants to an open-ended AI fact-checking system. Each claim is classified along five semantic dimensions (domain, epistemic form, verifiability, target entity, and temporal reference), producing a behavioral map of public verification demand. Three findings stand out. First, users range widely across topics but default to a narrow epistemic repertoire, overwhelmingly submitting simple descriptive claims about present-day observables. Second, roughly one in four requests concerns statements that cannot be empirically resolved, including moral judgments, speculative predictions, and subjective evaluations, revealing a systematic mismatch between what users seek from fact-checking tools and what such tools can deliver. Third, comparison with the FEVER benchmark dataset exposes sharp structural divergences across all five dimensions, indicating that standard evaluation corpora encode a synthetic claim environment that does not resemble real-world verification needs. These results reframe fact-checking as a demand-driven problem and identify where current AI systems and benchmarks are misaligned with the uncertainty people actually experience.

Related papers

• Machine Bullshit: Characterizing the Emergent Disregard for Truth in Large Language Models [57.834711966432685]
  Bullshit, as conceptualized by philosopher Harry Frankfurt, refers to statements made without regard to their truth value.<n>We introduce the Bullshit Index, a novel metric quantifying large language model's indifference to truth.<n>We observe prevalent machine bullshit in political contexts, with weasel words as the dominant strategy.
  arXiv  Detail & Related papers  (2025-07-10T07:11:57Z)
• FactLens: Benchmarking Fine-Grained Fact Verification [6.814173254027381]
  We advocate for a shift towards fine-grained verification, where complex claims are broken down into smaller sub-claims for individual verification.<n>We introduce FactLens, a benchmark for evaluating fine-grained fact verification, with metrics and automated evaluators of sub-claim quality.<n>Our results show alignment between automated FactLens evaluators and human judgments, and we discuss the impact of sub-claim characteristics on the overall verification performance.
  arXiv  Detail & Related papers  (2024-11-08T21:26:57Z)
• Missci: Reconstructing Fallacies in Misrepresented Science [84.32990746227385]
  Health-related misinformation on social networks can lead to poor decision-making and real-world dangers. Missci is a novel argumentation theoretical model for fallacious reasoning. We present Missci as a dataset to test the critical reasoning abilities of large language models.
  arXiv  Detail & Related papers  (2024-06-05T12:11:10Z)
• From Chaos to Clarity: Claim Normalization to Empower Fact-Checking [57.024192702939736]
  Claim Normalization (aka ClaimNorm) aims to decompose complex and noisy social media posts into more straightforward and understandable forms. We propose CACN, a pioneering approach that leverages chain-of-thought and claim check-worthiness estimation. Our experiments demonstrate that CACN outperforms several baselines across various evaluation measures.
  arXiv  Detail & Related papers  (2023-10-22T16:07:06Z)
• FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering [3.0401523614135333]
  A human fact-checker generally follows several logical steps to verify a verisimilitude claim. It is necessary to have an aspect-based (delineating which part(s) are true and which are false) explainable system. In this paper, we propose a 5W framework for question-answer-based fact explainability.
  arXiv  Detail & Related papers  (2023-05-07T16:52:21Z)
• Ask to Know More: Generating Counterfactual Explanations for Fake Claims [11.135087647482145]
  We propose elucidating fact checking predictions using counterfactual explanations to help people understand why a piece of news was identified as fake. In this work, generating counterfactual explanations for fake news involves three steps: asking good questions, finding contradictions, and reasoning appropriately. Results suggest that the proposed approach generates the most helpful explanations compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-10T04:42:00Z)
• Generating Literal and Implied Subquestions to Fact-check Complex Claims [64.81832149826035]
  We focus on decomposing a complex claim into a comprehensive set of yes-no subquestions whose answers influence the veracity of the claim. We present ClaimDecomp, a dataset of decompositions for over 1000 claims. We show that these subquestions can help identify relevant evidence to fact-check the full claim and derive the veracity through their answers.
  arXiv  Detail & Related papers  (2022-05-14T00:40:57Z)
• FaVIQ: FAct Verification from Information-seeking Questions [77.7067957445298]
  We construct a large-scale fact verification dataset called FaVIQ using information-seeking questions posed by real users. Our claims are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.
  arXiv  Detail & Related papers  (2021-07-05T17:31:44Z)
• AmbiFC: Fact-Checking Ambiguous Claims with Evidence [57.7091560922174]
  We present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. We analyze disagreements arising from ambiguity when comparing claims against evidence in AmbiFC. We develop models for predicting veracity handling this ambiguity via soft labels.
  arXiv  Detail & Related papers  (2021-04-01T17:40:08Z)
• Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness [1.2891210250935146]
  We propose a model prioritizing the claims based on their check-worthiness. We use BERT model with additional features including domain-specific controversial topics, word embeddings, and others.
  arXiv  Detail & Related papers  (2020-04-17T10:55:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.