That is a Known Lie: Detecting Previously Fact-Checked Claims
- URL: http://arxiv.org/abs/2005.06058v1
- Date: Tue, 12 May 2020 21:25:37 GMT
- Title: That is a Known Lie: Detecting Previously Fact-Checked Claims
- Authors: Shaden Shaar, Giovanni Da San Martino, Nikolay Babulkov, Preslav Nakov
- Abstract summary: A large number of fact-checked claims have been accumulated.
Politicians like to repeat their favorite statements, true or false, over and over again.
It is important to try to save this effort and to avoid wasting time on claims that have already been fact-checked.
- Score: 34.30218503006579
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The recent proliferation of "fake news" has triggered a number of responses,
most notably the emergence of several manual fact-checking initiatives. As a
result and over time, a large number of fact-checked claims have been
accumulated, which increases the likelihood that a new claim in social media or
a new statement by a politician might have already been fact-checked by some
trusted fact-checking organization, as viral claims often come back after a
while in social media, and politicians like to repeat their favorite
statements, true or false, over and over again. As manual fact-checking is very
time-consuming (and fully automatic fact-checking has credibility issues), it
is important to try to save this effort and to avoid wasting time on claims
that have already been fact-checked. Interestingly, despite the importance of
the task, it has been largely ignored by the research community so far. Here,
we aim to bridge this gap. In particular, we formulate the task and we discuss
how it relates to, but also differs from, previous work. We further create a
specialized dataset, which we release to the research community. Finally, we
present learning-to-rank experiments that demonstrate sizable improvements over
state-of-the-art retrieval and textual similarity approaches.
Related papers
- Fact-checking information from large language models can decrease headline discernment [6.814801748069122]
We investigate the impact of fact-checking information generated by a popular large language model on belief in, and sharing intent of, political news headlines.
We find that this information does not significantly improve participants' ability to discern headline accuracy or share accurate news.
Our findings highlight an important source of potential harm stemming from AI applications.
arXiv Detail & Related papers (2023-08-21T15:47:37Z) - Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for
Misinformation [67.69725605939315]
Misinformation emerges in times of uncertainty when credible information is limited.
This is challenging for NLP-based fact-checking as it relies on counter-evidence, which may not yet be available.
arXiv Detail & Related papers (2022-10-25T09:40:48Z) - CrowdChecked: Detecting Previously Fact-Checked Claims in Social Media [19.688259030184508]
We propose an end-to-end framework to learn from noisy data based on modified self-adaptive training.
Our experiments on the CLEF'21 CheckThat! test set show improvements over the state of the art by two points absolute.
arXiv Detail & Related papers (2022-10-10T06:05:52Z) - Generating Literal and Implied Subquestions to Fact-check Complex Claims [64.81832149826035]
We focus on decomposing a complex claim into a comprehensive set of yes-no subquestions whose answers influence the veracity of the claim.
We present ClaimDecomp, a dataset of decompositions for over 1000 claims.
We show that these subquestions can help identify relevant evidence to fact-check the full claim and derive the veracity through their answers.
arXiv Detail & Related papers (2022-05-14T00:40:57Z) - Can online attention signals help fact-checkers fact-check? [11.65819327593072]
This paper proposes a framework to study fact-checking with online attention signals.
We use this framework to conduct a preliminary study of a dataset of 879 COVID-19-related fact-checks done in 2020 by 81 international organizations.
arXiv Detail & Related papers (2021-09-20T06:51:59Z) - Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked
Claims in a Document [27.076320857009655]
Given an input document, it aims to detect all sentences that contain a claim that can be verified by some previously fact-checked claims.
The output is a re-ranked list of the document sentences, so that those that can be verified are ranked as high as possible.
Our analysis demonstrates the importance of modeling text similarity and stance, while also taking into account the veracity of the retrieved previously fact-checked claims.
arXiv Detail & Related papers (2021-09-14T13:46:52Z) - FaVIQ: FAct Verification from Information-seeking Questions [77.7067957445298]
We construct a large-scale fact verification dataset called FaVIQ using information-seeking questions posed by real users.
Our claims are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.
arXiv Detail & Related papers (2021-07-05T17:31:44Z) - The Role of Context in Detecting Previously Fact-Checked Claims [27.076320857009655]
We focus on claims made in a political debate, where context really matters.
We study the impact of modeling the context of the claim both on the source side, as well as on the target side, in the fact-checking explanation document.
arXiv Detail & Related papers (2021-04-15T12:39:37Z) - Generating Fact Checking Briefs [97.82546239639964]
We investigate how to increase the accuracy and efficiency of fact checking by providing information about the claim before performing the check.
We develop QABriefer, a model that generates a set of questions conditioned on the claim, searches the web for evidence, and generates answers.
We show that fact checking with briefs -- in particular QABriefs -- increases the accuracy of crowdworkers by 10% while slightly decreasing the time taken.
arXiv Detail & Related papers (2020-11-10T23:02:47Z) - Misinformation Has High Perplexity [55.47422012881148]
We propose to leverage the perplexity to debunk false claims in an unsupervised manner.
First, we extract reliable evidence from scientific and news sources according to sentence similarity to the claims.
Second, we prime a language model with the extracted evidence and finally evaluate the correctness of given claims based on the perplexity scores at debunking time.
arXiv Detail & Related papers (2020-06-08T15:13:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.