COVID-Fact: Fact Extraction and Verification of Real-World Claims on
COVID-19 Pandemic
- URL: http://arxiv.org/abs/2106.03794v1
- Date: Mon, 7 Jun 2021 16:59:46 GMT
- Title: COVID-Fact: Fact Extraction and Verification of Real-World Claims on
COVID-19 Pandemic
- Authors: Arkadiy Saakyan, Tuhin Chakrabarty, and Smaranda Muresan
- Abstract summary: We introduce a FEVER-like dataset COVID-Fact of $4,086$ claims concerning the COVID-19 pandemic.
The dataset contains claims, evidence for the claims, and contradictory claims refuted by the evidence.
- Score: 12.078052727772718
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We introduce a FEVER-like dataset COVID-Fact of $4,086$ claims concerning the
COVID-19 pandemic. The dataset contains claims, evidence for the claims, and
contradictory claims refuted by the evidence. Unlike previous approaches, we
automatically detect true claims and their source articles and then generate
counter-claims using automatic methods rather than employing human annotators.
Along with our constructed resource, we formally present the task of
identifying relevant evidence for the claims and verifying whether the evidence
refutes or supports a given claim. In addition to scientific claims, our data
contains simplified general claims from media sources, making it better suited
for detecting general misinformation regarding COVID-19. Our experiments
indicate that COVID-Fact will provide a challenging testbed for the development
of new systems and our approach will reduce the costs of building
domain-specific datasets for detecting misinformation.
Related papers
- Minimal Evidence Group Identification for Claim Verification [15.8357231063287]
We study the problem of identifying minimal evidence groups (MEGs) for claim verification.
We show that MEG identification can be reduced from Set Cover problem, based on entailment inference of whether a given evidence group provides full/partial support to a claim.
Our proposed approach achieves 18.4% and 34.8% absolute improvements on the WiCE and SciFact datasets.
arXiv Detail & Related papers (2024-04-24T01:44:09Z) - AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from
the Web [20.576644330553744]
We introduce AVeriTeC, a new dataset of 4,568 real-world claims covering fact-checks by 50 different organizations.
Each claim is annotated with question-answer pairs supported by evidence available online, as well as textual justifications explaining how the evidence combines to produce a verdict.
arXiv Detail & Related papers (2023-05-22T15:17:18Z) - Complex Claim Verification with Evidence Retrieved in the Wild [73.19998942259073]
We present the first fully automated pipeline to check real-world claims by retrieving raw evidence from the web.
Our pipeline includes five components: claim decomposition, raw document retrieval, fine-grained evidence retrieval, claim-focused summarization, and veracity judgment.
arXiv Detail & Related papers (2023-05-19T17:49:19Z) - WiCE: Real-World Entailment for Claims in Wikipedia [63.234352061821625]
We propose WiCE, a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia.
In addition to standard claim-level entailment, WiCE provides entailment judgments over sub-sentence units of the claim.
We show that real claims in our dataset involve challenging verification and retrieval problems that existing models fail to address.
arXiv Detail & Related papers (2023-03-02T17:45:32Z) - Generating Literal and Implied Subquestions to Fact-check Complex Claims [64.81832149826035]
We focus on decomposing a complex claim into a comprehensive set of yes-no subquestions whose answers influence the veracity of the claim.
We present ClaimDecomp, a dataset of decompositions for over 1000 claims.
We show that these subquestions can help identify relevant evidence to fact-check the full claim and derive the veracity through their answers.
arXiv Detail & Related papers (2022-05-14T00:40:57Z) - Generating Scientific Claims for Zero-Shot Scientific Fact Checking [54.62086027306609]
Automated scientific fact checking is difficult due to the complexity of scientific language and a lack of significant amounts of training data.
We propose scientific claim generation, the task of generating one or more atomic and verifiable claims from scientific sentences.
We also demonstrate its usefulness in zero-shot fact checking for biomedical claims.
arXiv Detail & Related papers (2022-03-24T11:29:20Z) - Synthetic Disinformation Attacks on Automated Fact Verification Systems [53.011635547834025]
We explore the sensitivity of automated fact-checkers to synthetic adversarial evidence in two simulated settings.
We show that these systems suffer significant performance drops against these attacks.
We discuss the growing threat of modern NLG systems as generators of disinformation.
arXiv Detail & Related papers (2022-02-18T19:01:01Z) - Robust Information Retrieval for False Claims with Distracting Entities
In Fact Extraction and Verification [2.624734563929267]
This paper shows that, compared with true claims, false claims more frequently contain irrelevant entities which can distract evidence retrieval model.
A BERT-based retrieval model made more mistakes in retrieving refuting evidence for false claims than supporting evidence for true claims.
arXiv Detail & Related papers (2021-12-10T17:11:50Z) - Misinformation Has High Perplexity [55.47422012881148]
We propose to leverage the perplexity to debunk false claims in an unsupervised manner.
First, we extract reliable evidence from scientific and news sources according to sentence similarity to the claims.
Second, we prime a language model with the extracted evidence and finally evaluate the correctness of given claims based on the perplexity scores at debunking time.
arXiv Detail & Related papers (2020-06-08T15:13:44Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.