DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking

• URL: http://arxiv.org/abs/2004.12864v1
• Date: Mon, 27 Apr 2020 15:18:49 GMT
• Title: DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking
• Authors: Christopher Hidey and Tuhin Chakrabarty and Tariq Alhindi and Siddharth Varia and Kriste Krstovski and Mona Diab and Smaranda Muresan
• Abstract summary: We show that current systems for fact-checking are vulnerable to three categories of realistic challenges for fact-checking. We present a system designed to be resilient to these "attacks" using multiple pointer networks for document selection. We find that in handling these attacks we obtain state-of-the-art results on FEVER, largely due to improved evidence retrieval.
• Score: 46.13738685855884
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increased focus on misinformation has spurred development of data and systems for detecting the veracity of a claim as well as retrieving authoritative evidence. The Fact Extraction and VERification (FEVER) dataset provides such a resource for evaluating end-to-end fact-checking, requiring retrieval of evidence from Wikipedia to validate a veracity prediction. We show that current systems for FEVER are vulnerable to three categories of realistic challenges for fact-checking -- multiple propositions, temporal reasoning, and ambiguity and lexical variation -- and introduce a resource with these types of claims. Then we present a system designed to be resilient to these "attacks" using multiple pointer networks for document selection and jointly modeling a sequence of evidence sentences and veracity relation predictions. We find that in handling these attacks we obtain state-of-the-art results on FEVER, largely due to improved evidence retrieval.

Related papers

• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification [22.785622371421876]
  We present a pioneering dataset for multi-hop explainable fact verification. With over 60,000 claims involving 2-hop and 3-hop reasoning, each is created by summarizing and modifying information from hyperlinked Wikipedia documents. We demonstrate a novel baseline system on our EX-FEVER dataset, showcasing document retrieval, explanation generation, and claim verification.
  arXiv  Detail & Related papers  (2023-10-15T06:46:15Z)
• Give Me More Details: Improving Fact-Checking with Latent Retrieval [58.706972228039604]
  Evidence plays a crucial role in automated fact-checking. Existing fact-checking systems either assume the evidence sentences are given or use the search snippets returned by the search engine. We propose to incorporate full text from source documents as evidence and introduce two enriched datasets.
  arXiv  Detail & Related papers  (2023-05-25T15:01:19Z)
• Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networks [20.282527436527765]
  We propose a unified Graph-based sEmantic structure mining framework with ConTRAstive Learning, namely GETRAL in short. We first model claims and evidences as graph-structured data to capture the long-distance semantic dependency. Then the fine-grained semantic representations are fed into the claim-evidence interaction module for predictions.
  arXiv  Detail & Related papers  (2022-10-11T14:54:37Z)
• GERE: Generative Evidence Retrieval for Fact Verification [57.78768817972026]
  We propose GERE, the first system that retrieves evidences in a generative fashion. The experimental results on the FEVER dataset show that GERE achieves significant improvements over the state-of-the-art baselines.
  arXiv  Detail & Related papers  (2022-04-12T03:49:35Z)
• Synthetic Disinformation Attacks on Automated Fact Verification Systems [53.011635547834025]
  We explore the sensitivity of automated fact-checkers to synthetic adversarial evidence in two simulated settings. We show that these systems suffer significant performance drops against these attacks. We discuss the growing threat of modern NLG systems as generators of disinformation.
  arXiv  Detail & Related papers  (2022-02-18T19:01:01Z)
• Mining Fine-grained Semantics via Graph Neural Networks for Evidence-based Fake News Detection [20.282527436527765]
  We propose a unified Graph-based sEmantic sTructure mining framework, namely GET in short. We model claims and evidences as graph-structured data and capture the long-distance semantic dependency. After obtaining contextual semantic information, our model reduces information redundancy by performing graph structure learning.
  arXiv  Detail & Related papers  (2022-01-18T11:28:36Z)
• Generating Fact Checking Explanations [52.879658637466605]
  A crucial piece of the puzzle that is still missing is to understand how to automate the most elaborate part of the process. This paper provides the first study of how these explanations can be generated automatically based on available claim context. Our results indicate that optimising both objectives at the same time, rather than training them separately, improves the performance of a fact checking system.
  arXiv  Detail & Related papers  (2020-04-13T05:23:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.