Does Self-Rationalization Improve Robustness to Spurious Correlations?
- URL: http://arxiv.org/abs/2210.13575v1
- Date: Mon, 24 Oct 2022 19:54:57 GMT
- Title: Does Self-Rationalization Improve Robustness to Spurious Correlations?
- Authors: Alexis Ross, Matthew E. Peters, Ana Marasovi\'c
- Abstract summary: We ask whether training models to self-rationalize can aid in their learning to solve tasks for the right reasons.
We evaluate robustness to spurious correlations in fine-tuned encoder-decoder and decoder-only models of six different sizes.
We find that while self-rationalization can improve robustness to spurious correlations in low-resource settings, it tends to hurt robustness in higher-resource settings.
- Score: 19.553357015260687
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Rationalization is fundamental to human reasoning and learning. NLP models
trained to produce rationales along with predictions, called
self-rationalization models, have been investigated for their interpretability
and utility to end-users. However, the extent to which training with
human-written rationales facilitates learning remains an under-explored
question. We ask whether training models to self-rationalize can aid in their
learning to solve tasks for the right reasons. Specifically, we evaluate how
training self-rationalization models with free-text rationales affects
robustness to spurious correlations in fine-tuned encoder-decoder and
decoder-only models of six different sizes. We evaluate robustness to spurious
correlations by measuring performance on 1) manually annotated challenge
datasets and 2) subsets of original test sets where reliance on spurious
correlations would fail to produce correct answers. We find that while
self-rationalization can improve robustness to spurious correlations in
low-resource settings, it tends to hurt robustness in higher-resource settings.
Furthermore, these effects depend on model family and size, as well as on
rationale content. Together, our results suggest that explainability can come
at the cost of robustness; thus, appropriate care should be taken when training
self-rationalizing models with the goal of creating more trustworthy models.
Related papers
- Improving Language Model Reasoning with Self-motivated Learning [60.779625789039486]
textitSelf-motivated Learning framework motivates the model itself to automatically generate rationales on existing datasets.
We train a reward model with the rank to evaluate the quality of rationales, and improve the performance of reasoning through reinforcement learning.
arXiv Detail & Related papers (2024-04-10T14:05:44Z) - Selective Learning: Towards Robust Calibration with Dynamic Regularization [79.92633587914659]
Miscalibration in deep learning refers to there is a discrepancy between the predicted confidence and performance.
We introduce Dynamic Regularization (DReg) which aims to learn what should be learned during training thereby circumventing the confidence adjusting trade-off.
arXiv Detail & Related papers (2024-02-13T11:25:20Z) - Seeing is not Believing: Robust Reinforcement Learning against Spurious
Correlation [57.351098530477124]
We consider one critical type of robustness against spurious correlation, where different portions of the state do not have correlations induced by unobserved confounders.
A model that learns such useless or even harmful correlation could catastrophically fail when the confounder in the test case deviates from the training one.
Existing robust algorithms that assume simple and unstructured uncertainty sets are therefore inadequate to address this challenge.
arXiv Detail & Related papers (2023-07-15T23:53:37Z) - ZARA: Improving Few-Shot Self-Rationalization for Small Language Models [29.755148112827502]
We present a novel approach, Zero-shot Augmentation of Rationale-Answer pairs (ZARA), to automatically construct pseudo-parallel data for self-training.
ZARA achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric.
arXiv Detail & Related papers (2023-05-12T10:07:12Z) - Less is More: Mitigate Spurious Correlations for Open-Domain Dialogue
Response Generation Models by Causal Discovery [52.95935278819512]
We conduct the first study on spurious correlations for open-domain response generation models based on a corpus CGDIALOG curated in our work.
Inspired by causal discovery algorithms, we propose a novel model-agnostic method for training and inference of response generation model.
arXiv Detail & Related papers (2023-03-02T06:33:48Z) - Explicit Tradeoffs between Adversarial and Natural Distributional
Robustness [48.44639585732391]
In practice, models need to enjoy both types of robustness to ensure reliability.
In this work, we show that in fact, explicit tradeoffs exist between adversarial and natural distributional robustness.
arXiv Detail & Related papers (2022-09-15T19:58:01Z) - Can Rationalization Improve Robustness? [39.741059642044874]
We investigate whether neural NLP models can provide robustness to adversarial attacks in addition to their interpretable nature.
We generate various types of 'AddText' attacks for both token and sentence-level rationalization tasks.
Our experiments reveal that the rationale models show the promise to improve robustness, while they struggle in certain scenarios.
arXiv Detail & Related papers (2022-04-25T17:02:42Z) - Enhancing Model Robustness and Fairness with Causality: A Regularization
Approach [15.981724441808147]
Recent work has raised concerns on the risk of spurious correlations and unintended biases in machine learning models.
We propose a simple and intuitive regularization approach to integrate causal knowledge during model training.
We build a predictive model that relies more on causal features and less on non-causal features.
arXiv Detail & Related papers (2021-10-03T02:49:33Z) - Measuring Association Between Labels and Free-Text Rationales [60.58672852655487]
In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance.
We demonstrate that pipelines, existing models for faithful extractive rationalization on information-extraction style tasks, do not extend as reliably to "reasoning" tasks requiring free-text rationales.
We turn to models that jointly predict and rationalize, a class of widely used high-performance models for free-text rationalization whose faithfulness is not yet established.
arXiv Detail & Related papers (2020-10-24T03:40:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.