A Multi-Level Attention Model for Evidence-Based Fact Checking
- URL: http://arxiv.org/abs/2106.00950v1
- Date: Wed, 2 Jun 2021 05:40:12 GMT
- Title: A Multi-Level Attention Model for Evidence-Based Fact Checking
- Authors: Canasai Kruengkrai, Junichi Yamagishi, Xin Wang
- Abstract summary: We present a simple model that can be trained on sequence structures.
Results on a large-scale dataset for Fact Extraction and VERification show that our model outperforms the graph-based approaches.
- Score: 58.95413968110558
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Evidence-based fact checking aims to verify the truthfulness of a claim
against evidence extracted from textual sources. Learning a representation that
effectively captures relations between a claim and evidence can be challenging.
Recent state-of-the-art approaches have developed increasingly sophisticated
models based on graph structures. We present a simple model that can be trained
on sequence structures. Our model enables inter-sentence attentions at
different levels and can benefit from joint training. Results on a large-scale
dataset for Fact Extraction and VERification (FEVER) show that our model
outperforms the graph-based approaches and yields 1.09% and 1.42% improvements
in label accuracy and FEVER score, respectively, over the best published model.
Related papers
- ZeFaV: Boosting Large Language Models for Zero-shot Fact Verification [2.6874004806796523]
ZeFaV is a zero-shot based fact-checking verification framework to enhance the performance on fact verification task of large language models.
We conducted empirical experiments to evaluate our approach on two multi-hop fact-checking datasets including HoVer and FEVEROUS.
arXiv Detail & Related papers (2024-11-18T02:35:15Z) - Entity-level Factual Adaptiveness of Fine-tuning based Abstractive
Summarization Models [31.84120883461332]
We analyze the robustness of fine-tuning based summarization models to the knowledge conflict.
We introduce a controllable counterfactual data augmentation method.
arXiv Detail & Related papers (2024-02-23T07:53:39Z) - A Comprehensive Evaluation and Analysis Study for Chinese Spelling Check [53.152011258252315]
We show that using phonetic and graphic information reasonably is effective for Chinese Spelling Check.
Models are sensitive to the error distribution of the test set, which reflects the shortcomings of models.
The commonly used benchmark, SIGHAN, can not reliably evaluate models' performance.
arXiv Detail & Related papers (2023-07-25T17:02:38Z) - Preserving Knowledge Invariance: Rethinking Robustness Evaluation of
Open Information Extraction [50.62245481416744]
We present the first benchmark that simulates the evaluation of open information extraction models in the real world.
We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique.
By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
arXiv Detail & Related papers (2023-05-23T12:05:09Z) - A Graph-Enhanced Click Model for Web Search [67.27218481132185]
We propose a novel graph-enhanced click model (GraphCM) for web search.
We exploit both intra-session and inter-session information for the sparsity and cold-start problems.
arXiv Detail & Related papers (2022-06-17T08:32:43Z) - A Relational Model for One-Shot Classification [80.77724423309184]
We show that a deep learning model with built-in inductive bias can bring benefits to sample-efficient learning, without relying on extensive data augmentation.
The proposed one-shot classification model performs relational matching of a pair of inputs in the form of local and pairwise attention.
arXiv Detail & Related papers (2021-11-08T07:53:12Z) - SAIS: Supervising and Augmenting Intermediate Steps for Document-Level
Relation Extraction [51.27558374091491]
We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction.
Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
arXiv Detail & Related papers (2021-09-24T17:37:35Z) - Integrating Pattern- and Fact-based Fake News Detection via Model
Preference Learning [6.92027612631023]
We study the problem of integrating pattern- and fact-based models into one framework.
We build a Preference-aware Fake News Detection Framework (Pref-FEND), which learns the respective preferences of pattern- and fact-based models for joint detection.
arXiv Detail & Related papers (2021-09-23T12:28:55Z) - The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning [25.85044477227461]
Models that are more accurate on the out-of-distribution data relative to this baseline exhibit "effective robustness"
We find that models pre-trained on larger datasets exhibit effective robustness during training that vanishes at convergence.
We discuss several strategies for scaling effective robustness to the high-accuracy regime to improve the out-of-distribution accuracy of state-of-the-art models.
arXiv Detail & Related papers (2021-06-30T06:21:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.