Pathologies of Pre-trained Language Models in Few-shot Fine-tuning
- URL: http://arxiv.org/abs/2204.08039v1
- Date: Sun, 17 Apr 2022 15:55:18 GMT
- Title: Pathologies of Pre-trained Language Models in Few-shot Fine-tuning
- Authors: Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah, Yangfeng Ji
- Abstract summary: We show that pre-trained language models with few examples show strong prediction bias across labels.
Although few-shot fine-tuning can mitigate the prediction bias, our analysis shows models gain performance improvement by capturing non-task-related features.
These observations alert that pursuing model performance with fewer examples may incur pathological prediction behavior.
- Score: 50.3686606679048
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Although adapting pre-trained language models with few examples has shown
promising performance on text classification, there is a lack of understanding
of where the performance gain comes from. In this work, we propose to answer
this question by interpreting the adaptation behavior using post-hoc
explanations from model predictions. By modeling feature statistics of
explanations, we discover that (1) without fine-tuning, pre-trained models
(e.g. BERT and RoBERTa) show strong prediction bias across labels; (2) although
few-shot fine-tuning can mitigate the prediction bias and demonstrate promising
prediction performance, our analysis shows models gain performance improvement
by capturing non-task-related features (e.g. stop words) or shallow data
patterns (e.g. lexical overlaps). These observations alert that pursuing model
performance with fewer examples may incur pathological prediction behavior,
which requires further sanity check on model predictions and careful design in
model evaluations in few-shot fine-tuning.
Related papers
- Predictive Churn with the Set of Good Models [64.05949860750235]
We study the effect of conflicting predictions over the set of near-optimal machine learning models.
We present theoretical results on the expected churn between models within the Rashomon set.
We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
arXiv Detail & Related papers (2024-02-12T16:15:25Z) - What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement [38.93348195407474]
Language models deployed in the wild make errors.
Updating the model with the corrected error instances causes catastrophic forgetting.
We propose a partially interpretable forecasting model based on the observation that changes in pre-softmax logit scores of pretraining examples resemble that of online learned examples.
arXiv Detail & Related papers (2024-02-02T19:43:15Z) - Training Trajectories of Language Models Across Scales [99.38721327771208]
Scaling up language models has led to unprecedented performance gains.
How do language models of different sizes learn during pre-training?
Why do larger language models demonstrate more desirable behaviors?
arXiv Detail & Related papers (2022-12-19T19:16:29Z) - Performance Prediction Under Dataset Shift [1.1602089225841632]
We study the generalization capabilities of various performance prediction models to new domains by learning on generated synthetic perturbations.
We propose a natural and effortless uncertainty estimation of the predicted accuracy that ensures reliable use of performance predictors.
arXiv Detail & Related papers (2022-06-21T19:40:58Z) - Training Deep Models to be Explained with Fewer Examples [40.58343220792933]
We train prediction and explanation models simultaneously with a sparse regularizer for reducing the number of examples.
Experiments using several datasets demonstrate that the proposed method improves faithfulness while keeping the predictive performance.
arXiv Detail & Related papers (2021-12-07T05:39:21Z) - Instance-Based Neural Dependency Parsing [56.63500180843504]
We develop neural models that possess an interpretable inference process for dependency parsing.
Our models adopt instance-based inference, where dependency edges are extracted and labeled by comparing them to edges in a training set.
arXiv Detail & Related papers (2021-09-28T05:30:52Z) - Avoiding Inference Heuristics in Few-shot Prompt-based Finetuning [57.4036085386653]
We show that prompt-based models for sentence pair classification tasks still suffer from a common pitfall of adopting inferences based on lexical overlap.
We then show that adding a regularization that preserves pretraining weights is effective in mitigating this destructive tendency of few-shot finetuning.
arXiv Detail & Related papers (2021-09-09T10:10:29Z) - Translation Error Detection as Rationale Extraction [36.616561917049076]
We study the behaviour of state-of-the-art sentence-level QE models and show that explanations can indeed be used to detect translation errors.
We introduce a novel semi-supervised method for word-level QE and (ii) propose to use the QE task as a new benchmark for evaluating the plausibility of feature attribution.
arXiv Detail & Related papers (2021-08-27T09:35:14Z) - On the Lack of Robust Interpretability of Neural Text Classifiers [14.685352584216757]
We assess the robustness of interpretations of neural text classifiers based on pretrained Transformer encoders.
Both tests show surprising deviations from expected behavior, raising questions about the extent of insights that practitioners may draw from interpretations.
arXiv Detail & Related papers (2021-06-08T18:31:02Z) - Explaining and Improving Model Behavior with k Nearest Neighbor
Representations [107.24850861390196]
We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions.
We show that kNN representations are effective at uncovering learned spurious associations.
Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
arXiv Detail & Related papers (2020-10-18T16:55:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.