Measuring and Improving Attentiveness to Partial Inputs with
Counterfactuals
- URL: http://arxiv.org/abs/2311.09605v1
- Date: Thu, 16 Nov 2023 06:27:35 GMT
- Title: Measuring and Improving Attentiveness to Partial Inputs with
Counterfactuals
- Authors: Yanai Elazar, Bhargavi Paranjape, Hao Peng, Sarah Wiegreffe, Khyathi
Raghavi, Vivek Srikumar, Sameer Singh, Noah A. Smith
- Abstract summary: We propose a new evaluation method, Counterfactual Attentiveness Test (CAT)
CAT uses counterfactuals by replacing part of the input with its counterpart from a different example, expecting an attentive model to change its prediction.
We show that GPT3 becomes less attentive with an increased number of demonstrations, while its accuracy on the test data improves.
- Score: 95.5442607785241
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The inevitable appearance of spurious correlations in training datasets hurts
the generalization of NLP models on unseen data. Previous work has found that
datasets with paired inputs are prone to correlations between a specific part
of the input (e.g., the hypothesis in NLI) and the label; consequently, models
trained only on those outperform chance. Are these correlations picked up by
models trained on the full input data? To address this question, we propose a
new evaluation method, Counterfactual Attentiveness Test (CAT). CAT uses
counterfactuals by replacing part of the input with its counterpart from a
different example (subject to some restrictions), expecting an attentive model
to change its prediction. Using CAT, we systematically investigate established
supervised and in-context learning models on ten datasets spanning four tasks:
natural language inference, reading comprehension, paraphrase detection, and
visual & language reasoning. CAT reveals that reliance on such correlations is
mainly data-dependent. Surprisingly, we find that GPT3 becomes less attentive
with an increased number of demonstrations, while its accuracy on the test data
improves. Our results demonstrate that augmenting training or demonstration
data with counterfactuals is effective in improving models' attentiveness. We
show that models' attentiveness measured by CAT reveals different conclusions
from solely measuring correlations in data.
Related papers
- The Mirrored Influence Hypothesis: Efficient Data Influence Estimation by Harnessing Forward Passes [30.30769701138665]
We introduce and explore the Mirrored Influence Hypothesis, highlighting a reciprocal nature of influence between training and test data.
Specifically, it suggests that evaluating the influence of training data on test predictions can be reformulated as an equivalent, yet inverse problem.
We introduce a new method for estimating the influence of training data, which requires calculating gradients for specific test samples, paired with a forward pass for each training point.
arXiv Detail & Related papers (2024-02-14T03:43:05Z) - Towards Better Modeling with Missing Data: A Contrastive Learning-based
Visual Analytics Perspective [7.577040836988683]
Missing data can pose a challenge for machine learning (ML) modeling.
Current approaches are categorized into feature imputation and label prediction.
This study proposes a Contrastive Learning framework to model observed data with missing values.
arXiv Detail & Related papers (2023-09-18T13:16:24Z) - Stubborn Lexical Bias in Data and Models [50.79738900885665]
We use a new statistical method to examine whether spurious patterns in data appear in models trained on the data.
We apply an optimization approach to *reweight* the training data, reducing thousands of spurious correlations.
Surprisingly, though this method can successfully reduce lexical biases in the training data, we still find strong evidence of corresponding bias in the trained models.
arXiv Detail & Related papers (2023-06-03T20:12:27Z) - Measuring Causal Effects of Data Statistics on Language Model's
`Factual' Predictions [59.284907093349425]
Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models.
We provide a language for describing how training data influences predictions, through a causal framework.
Our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone.
arXiv Detail & Related papers (2022-07-28T17:36:24Z) - Impact of Pretraining Term Frequencies on Few-Shot Reasoning [51.990349528930125]
We investigate how well pretrained language models reason with terms that are less frequent in the pretraining data.
We measure the strength of this correlation for a number of GPT-based language models on various numerical deduction tasks.
Although LMs exhibit strong performance at few-shot numerical reasoning tasks, our results raise the question of how much models actually generalize beyond pretraining data.
arXiv Detail & Related papers (2022-02-15T05:43:54Z) - Conformal prediction for the design problem [72.14982816083297]
In many real-world deployments of machine learning, we use a prediction algorithm to choose what data to test next.
In such settings, there is a distinct type of distribution shift between the training and test data.
We introduce a method to quantify predictive uncertainty in such settings.
arXiv Detail & Related papers (2022-02-08T02:59:12Z) - Counterfactual Adversarial Learning with Representation Interpolation [11.843735677432166]
We introduce Counterfactual Adrial Training framework to tackle the problem from aversa causality perspective.
Experiments demonstrate that CAT achieves substantial performance improvement over SOTA across different downstream tasks.
arXiv Detail & Related papers (2021-09-10T09:23:08Z) - Explaining and Improving Model Behavior with k Nearest Neighbor
Representations [107.24850861390196]
We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions.
We show that kNN representations are effective at uncovering learned spurious associations.
Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
arXiv Detail & Related papers (2020-10-18T16:55:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.