RerrFact: Reduced Evidence Retrieval Representations for Scientific Claim Verification

• URL: http://arxiv.org/abs/2202.02646v1
• Date: Sat, 5 Feb 2022 21:52:45 GMT
• Title: RerrFact: Reduced Evidence Retrieval Representations for Scientific Claim Verification
• Authors: Ashish Rana, Deepanshu Khanna, Muskaan Singh, Tirthankar Ghosal, Harpreet Singh and Prashant Singh Rana
• Abstract summary: We propose a modular approach that sequentially carries out binary classification for every prediction subtask. We carry out two-step stance predictions that first differentiate non-relevant rationales and then identify supporting or refuting rationales for a given claim. Experimentally, our system RerrFact with no fine-tuning, simple design, and a fraction of model parameters fairs competitively on the leaderboard.
• Score: 4.052777228128475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exponential growth in digital information outlets and the race to publish has made scientific misinformation more prevalent than ever. However, the task to fact-verify a given scientific claim is not straightforward even for researchers. Scientific claim verification requires in-depth knowledge and great labor from domain experts to substantiate supporting and refuting evidence from credible scientific sources. The SciFact dataset and corresponding task provide a benchmarking leaderboard to the community to develop automatic scientific claim verification systems via extracting and assimilating relevant evidence rationales from source abstracts. In this work, we propose a modular approach that sequentially carries out binary classification for every prediction subtask as in the SciFact leaderboard. Our simple classifier-based approach uses reduced abstract representations to retrieve relevant abstracts. These are further used to train the relevant rationale-selection model. Finally, we carry out two-step stance predictions that first differentiate non-relevant rationales and then identify supporting or refuting rationales for a given claim. Experimentally, our system RerrFact with no fine-tuning, simple design, and a fraction of model parameters fairs competitively on the leaderboard against large-scale, modular, and joint modeling approaches. We make our codebase available at https://github.com/ashishrana160796/RerrFact.

Related papers

• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• Missci: Reconstructing Fallacies in Misrepresented Science [84.32990746227385]
  Health-related misinformation on social networks can lead to poor decision-making and real-world dangers. Missci is a novel argumentation theoretical model for fallacious reasoning. We present Missci as a dataset to test the critical reasoning abilities of large language models.
  arXiv  Detail & Related papers  (2024-06-05T12:11:10Z)
• Large Language Models for Automated Open-domain Scientific Hypotheses Discovery [50.40483334131271]
  This work proposes the first dataset for social science academic hypotheses discovery. Unlike previous settings, the new dataset requires (1) using open-domain data (raw web corpus) as observations; and (2) proposing hypotheses even new to humanity. A multi- module framework is developed for the task, including three different feedback mechanisms to boost performance.
  arXiv  Detail & Related papers  (2023-09-06T05:19:41Z)
• SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables [68.76415918462418]
  We present SCITAB, a challenging evaluation dataset consisting of 1.2K expert-verified scientific claims. Through extensive evaluations, we demonstrate that SCITAB poses a significant challenge to state-of-the-art models. Our analysis uncovers several unique challenges posed by SCITAB, including table grounding, claim ambiguity, and compositional reasoning.
  arXiv  Detail & Related papers  (2023-05-22T16:13:50Z)
• ExClaim: Explainable Neural Claim Verification Using Rationalization [8.369720566612111]
  ExClaim attempts to provide an explainable claim verification system with foundational evidence. Inspired by the legal system, ExClaim leverages rationalization to provide a verdict for the claim. Statistical and Explainable AI (XAI) evaluations are conducted to ensure valid and trustworthy outcomes.
  arXiv  Detail & Related papers  (2023-01-21T08:26:27Z)
• SciFact-Open: Towards open-domain scientific claim verification [61.288725621156864]
  We present SciFact-Open, a new test collection designed to evaluate the performance of scientific claim verification systems. We collect evidence for scientific claims by pooling and annotating the top predictions of four state-of-the-art scientific claim verification models. We find that systems developed on smaller corpora struggle to generalize to SciFact-Open, exhibiting performance drops of at least 15 F1.
  arXiv  Detail & Related papers  (2022-10-25T05:45:00Z)
• Adversarial Contrastive Learning for Evidence-aware Fake News Detection with Graph Neural Networks [20.282527436527765]
  We propose a unified Graph-based sEmantic structure mining framework with ConTRAstive Learning, namely GETRAL in short. We first model claims and evidences as graph-structured data to capture the long-distance semantic dependency. Then the fine-grained semantic representations are fed into the claim-evidence interaction module for predictions.
  arXiv  Detail & Related papers  (2022-10-11T14:54:37Z)
• Mining Fine-grained Semantics via Graph Neural Networks for Evidence-based Fake News Detection [20.282527436527765]
  We propose a unified Graph-based sEmantic sTructure mining framework, namely GET in short. We model claims and evidences as graph-structured data and capture the long-distance semantic dependency. After obtaining contextual semantic information, our model reduces information redundancy by performing graph structure learning.
  arXiv  Detail & Related papers  (2022-01-18T11:28:36Z)
• Abstract, Rationale, Stance: A Joint Model for Scientific Claim Verification [18.330265729989843]
  We propose an approach, named as ARSJoint, that jointly learns the modules for the three tasks with a machine reading comprehension framework. The experimental results on the benchmark dataset SciFact show that our approach outperforms the existing works.
  arXiv  Detail & Related papers  (2021-09-13T10:07:26Z)
• Fact or Fiction: Verifying Scientific Claims [53.29101835904273]
  We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that SUPPORTS or REFUTES a given scientific claim. We construct SciFact, a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. We show that our system is able to verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus.
  arXiv  Detail & Related papers  (2020-04-30T17:22:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.