Table-based Fact Verification with Salience-aware Learning
- URL: http://arxiv.org/abs/2109.04053v1
- Date: Thu, 9 Sep 2021 06:18:46 GMT
- Title: Table-based Fact Verification with Salience-aware Learning
- Authors: Fei Wang, Kexuan Sun, Jay Pujara, Pedro Szekely, Muhao Chen
- Abstract summary: Salience estimation allows enhanced learning of fact verification from two perspectives.
Experimental results on TabFact show the effective improvement by the proposed salience-aware learning techniques.
- Score: 20.72341939868327
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Tables provide valuable knowledge that can be used to verify textual
statements. While a number of works have considered table-based fact
verification, direct alignments of tabular data with tokens in textual
statements are rarely available. Moreover, training a generalized fact
verification model requires abundant labeled training data. In this paper, we
propose a novel system to address these problems. Inspired by counterfactual
causality, our system identifies token-level salience in the statement with
probing-based salience estimation. Salience estimation allows enhanced learning
of fact verification from two perspectives. From one perspective, our system
conducts masked salient token prediction to enhance the model for alignment and
reasoning between the table and the statement. From the other perspective, our
system applies salience-aware data augmentation to generate a more diverse set
of training instances by replacing non-salient terms. Experimental results on
TabFact show the effective improvement by the proposed salience-aware learning
techniques, leading to the new SOTA performance on the benchmark. Our code is
publicly available at https://github.com/luka-group/Salience-aware-Learning .
Related papers
- Tabular Transfer Learning via Prompting LLMs [52.96022335067357]
We propose a novel framework, Prompt to Transfer (P2T), that utilizes unlabeled (or heterogeneous) source data with large language models (LLMs)
P2T identifies a column feature in a source dataset that is strongly correlated with a target task feature to create examples relevant to the target task, thus creating pseudo-demonstrations for prompts.
arXiv Detail & Related papers (2024-08-09T11:30:52Z) - LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback [71.95402654982095]
We propose Math-Minos, a natural language feedback-enhanced verifier.
Our experiments reveal that a small set of natural language feedback can significantly boost the performance of the verifier.
arXiv Detail & Related papers (2024-06-20T06:42:27Z) - Making Pre-trained Language Models Great on Tabular Prediction [50.70574370855663]
The transferability of deep neural networks (DNNs) has made significant progress in image and language processing.
We present TP-BERTa, a specifically pre-trained LM for tabular data prediction.
A novel relative magnitude tokenization converts scalar numerical feature values to finely discrete, high-dimensional tokens, and an intra-feature attention approach integrates feature values with the corresponding feature names.
arXiv Detail & Related papers (2024-03-04T08:38:56Z) - Textual Entailment for Effective Triple Validation in Object Prediction [4.94309218465563]
We propose to use textual entailment to validate facts extracted from language models through cloze statements.
Our results show that triple validation based on textual entailment improves language model predictions in different training regimes.
arXiv Detail & Related papers (2024-01-29T16:50:56Z) - M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [103.6153593636399]
We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning)
It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario.
Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
arXiv Detail & Related papers (2023-03-09T09:05:47Z) - STUNT: Few-shot Tabular Learning with Self-generated Tasks from
Unlabeled Tables [64.0903766169603]
We propose a framework for few-shot semi-supervised learning, coined Self-generated Tasks from UNlabeled Tables (STUNT)
Our key idea is to self-generate diverse few-shot tasks by treating randomly chosen columns as a target label.
We then employ a meta-learning scheme to learn generalizable knowledge with the constructed tasks.
arXiv Detail & Related papers (2023-03-02T02:37:54Z) - Graph Reasoning with Context-Aware Linearization for Interpretable Fact
Extraction and Verification [18.80709296426521]
This paper presents an end-to-end system for fact extraction and verification using textual and tabular evidence.
Our best performing system achieves a FEVEROUS score of 0.23 and 53% label accuracy on the blind test data.
arXiv Detail & Related papers (2021-09-25T12:05:28Z) - Exploring Decomposition for Table-based Fact Verification [18.584226291619217]
We improve fact verification by decomposing complex statements into simpler subproblems.
Our proposed approach achieves the new state-of-the-art performance, an 82.7% accuracy, on the TabFact benchmark.
arXiv Detail & Related papers (2021-09-22T20:15:05Z) - Learning Better Representation for Tables by Self-Supervised Tasks [23.69766883380125]
We propose two self-supervised tasks, Number Ordering and Significance Ordering, to help to learn better table representation.
We test our methods on the widely used dataset ROTOWIRE which consists of NBA game statistic and related news.
arXiv Detail & Related papers (2020-10-15T09:03:38Z) - Learning What Makes a Difference from Counterfactual Examples and
Gradient Supervision [57.14468881854616]
We propose an auxiliary training objective that improves the generalization capabilities of neural networks.
We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task.
Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
arXiv Detail & Related papers (2020-04-20T02:47:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.