FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering

• URL: http://arxiv.org/abs/2305.04329v2
• Date: Sun, 28 May 2023 16:33:06 GMT
• Title: FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering
• Authors: Anku Rani, S.M Towhidul Islam Tonmoy, Dwip Dalal, Shreya Gautam, Megha Chakraborty, Aman Chadha, Amit Sheth, Amitava Das
• Abstract summary: A human fact-checker generally follows several logical steps to verify a verisimilitude claim. It is necessary to have an aspect-based (delineating which part(s) are true and which are false) explainable system. In this paper, we propose a 5W framework for question-answer-based fact explainability.
• Score: 3.0401523614135333
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic fact verification has received significant attention recently. Contemporary automatic fact-checking systems focus on estimating truthfulness using numerical scores which are not human-interpretable. A human fact-checker generally follows several logical steps to verify a verisimilitude claim and conclude whether its truthful or a mere masquerade. Popular fact-checking websites follow a common structure for fact categorization such as half true, half false, false, pants on fire, etc. Therefore, it is necessary to have an aspect-based (delineating which part(s) are true and which are false) explainable system that can assist human fact-checkers in asking relevant questions related to a fact, which can then be validated separately to reach a final verdict. In this paper, we propose a 5W framework (who, what, when, where, and why) for question-answer-based fact explainability. To that end, we present a semi-automatically generated dataset called FACTIFY-5WQA, which consists of 391, 041 facts along with relevant 5W QAs - underscoring our major contribution to this paper. A semantic role labeling system has been utilized to locate 5Ws, which generates QA pairs for claims using a masked language model. Finally, we report a baseline QA system to automatically locate those answers from evidence documents, which can serve as a baseline for future research in the field. Lastly, we propose a robust fact verification system that takes paraphrased claims and automatically validates them. The dataset and the baseline model are available at https: //github.com/ankuranii/acl-5W-QA

Related papers

• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• Overview of Factify5WQA: Fact Verification through 5W Question-Answering [25.679348027485254]
  The Factify5WQA task aims to increase research towards automated fake news detection. The best performing team posted an accuracy of 69.56%, which is a near 35% improvement over the baseline.
  arXiv  Detail & Related papers  (2024-10-05T17:28:18Z)
• QACHECK: A Demonstration System for Question-Guided Multi-Hop Fact-Checking [68.06355980166053]
  We propose the Question-guided Multi-hop Fact-Checking (QACHECK) system. It guides the model's reasoning process by asking a series of questions critical for verifying a claim. It provides the source of evidence supporting each question, fostering a transparent, explainable, and user-friendly fact-checking process.
  arXiv  Detail & Related papers  (2023-10-11T15:51:53Z)
• Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer? [43.03399918557937]
  In real-world applications, users often ask questions that don't have a definitive answer. We introduce QnotA, a dataset consisting of five different categories of questions that don't have definitive answers. With this data, we formulate three evaluation tasks that test a system's ability to 'identify', 'distinguish', and 'justify' QnotA questions. We show that even SOTA models including GPT-3 and Flan T5 do not fare well on these tasks and lack considerably behind the human performance baseline.
  arXiv  Detail & Related papers  (2023-09-08T23:12:03Z)
• MythQA: Query-Based Large-Scale Check-Worthy Claim Detection through Multi-Answer Open-Domain Question Answering [8.70509665552136]
  Check-worthy claim detection aims at providing plausible misinformation to downstream fact-checking systems or human experts to check. Many efforts have been put into how to identify check-worthy claims from a small scale of pre-collected claims, but how to efficiently detect check-worthy claims directly from a large-scale information source, such as Twitter, remains underexplored. We introduce MythQA, a new multi-answer open-domain question answering(QA) task that involves contradictory stance mining for query-based large-scale check-worthy claim detection.
  arXiv  Detail & Related papers  (2023-07-21T18:35:24Z)
• ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning [63.77667876176978]
  Large language models show improved downstream task interpretability when prompted to generate step-by-step reasoning to justify their final answers. These reasoning steps greatly improve model interpretability and verification, but objectively studying their correctness is difficult. We present ROS, a suite of interpretable, unsupervised automatic scores that improve and extend previous text generation evaluation metrics.
  arXiv  Detail & Related papers  (2022-12-15T15:52:39Z)
• Ask to Know More: Generating Counterfactual Explanations for Fake Claims [11.135087647482145]
  We propose elucidating fact checking predictions using counterfactual explanations to help people understand why a piece of news was identified as fake. In this work, generating counterfactual explanations for fake news involves three steps: asking good questions, finding contradictions, and reasoning appropriately. Results suggest that the proposed approach generates the most helpful explanations compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-10T04:42:00Z)
• Generating Literal and Implied Subquestions to Fact-check Complex Claims [64.81832149826035]
  We focus on decomposing a complex claim into a comprehensive set of yes-no subquestions whose answers influence the veracity of the claim. We present ClaimDecomp, a dataset of decompositions for over 1000 claims. We show that these subquestions can help identify relevant evidence to fact-check the full claim and derive the veracity through their answers.
  arXiv  Detail & Related papers  (2022-05-14T00:40:57Z)
• FaVIQ: FAct Verification from Information-seeking Questions [77.7067957445298]
  We construct a large-scale fact verification dataset called FaVIQ using information-seeking questions posed by real users. Our claims are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification.
  arXiv  Detail & Related papers  (2021-07-05T17:31:44Z)
• Zero-shot Fact Verification by Claim Generation [85.27523983027471]
  We develop QACG, a framework for training a robust fact verification model. We use automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia. In a zero-shot scenario, QACG improves a RoBERTa model's F1 from 50% to 77%, equivalent in performance to 2K+ manually-curated examples.
  arXiv  Detail & Related papers  (2021-05-31T03:13:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.