Geometry matters: Exploring language examples at the decision boundary
- URL: http://arxiv.org/abs/2010.07212v3
- Date: Thu, 28 Oct 2021 14:10:42 GMT
- Title: Geometry matters: Exploring language examples at the decision boundary
- Authors: Debajyoti Datta, Shashwat Kumar, Laura Barnes, Tom Fletcher
- Abstract summary: BERT, CNN and fasttext are susceptible to word substitutions in high difficulty examples.
On YelpReviewPolarity we observe a correlation coefficient of -0.4 between resilience to perturbations and the difficulty score.
Our approach is simple, architecture agnostic and can be used to study the fragilities of text classification models.
- Score: 2.7249290070320034
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: A growing body of recent evidence has highlighted the limitations of natural
language processing (NLP) datasets and classifiers. These include the presence
of annotation artifacts in datasets, classifiers relying on shallow features
like a single word (e.g., if a movie review has the word "romantic", the review
tends to be positive), or unnecessary words (e.g., learning a proper noun to
classify a movie as positive or negative). The presence of such artifacts has
subsequently led to the development of challenging datasets to force the model
to generalize better. While a variety of heuristic strategies, such as
counterfactual examples and contrast sets, have been proposed, the theoretical
justification about what makes these examples difficult for the classifier is
often lacking or unclear. In this paper, using tools from information geometry,
we propose a theoretical way to quantify the difficulty of an example in NLP.
Using our approach, we explore difficult examples for several deep learning
architectures. We discover that both BERT, CNN and fasttext are susceptible to
word substitutions in high difficulty examples. These classifiers tend to
perform poorly on the FIM test set. (generated by sampling and perturbing
difficult examples, with accuracy dropping below 50%). We replicate our
experiments on 5 NLP datasets (YelpReviewPolarity, AGNEWS, SogouNews,
YelpReviewFull and Yahoo Answers). On YelpReviewPolarity we observe a
correlation coefficient of -0.4 between resilience to perturbations and the
difficulty score. Similarly we observe a correlation of 0.35 between the
difficulty score and the empirical success probability of random substitutions.
Our approach is simple, architecture agnostic and can be used to study the
fragilities of text classification models. All the code used will be made
publicly available, including a tool to explore the difficult examples for
other datasets.
Related papers
- Detrimental Contexts in Open-Domain Question Answering [9.059854023578508]
We analyze how passages can have a detrimental effect on retrieve-then-read architectures used in question answering.
Our findings demonstrate that model accuracy can be improved by 10% on two popular QA datasets by filtering out detrimental passages.
arXiv Detail & Related papers (2023-10-27T11:45:16Z) - Understanding and Mitigating Spurious Correlations in Text
Classification with Neighborhood Analysis [69.07674653828565]
Machine learning models have a tendency to leverage spurious correlations that exist in the training set but may not hold true in general circumstances.
In this paper, we examine the implications of spurious correlations through a novel perspective called neighborhood analysis.
We propose a family of regularization methods, NFL (doN't Forget your Language) to mitigate spurious correlations in text classification.
arXiv Detail & Related papers (2023-05-23T03:55:50Z) - On the Blind Spots of Model-Based Evaluation Metrics for Text Generation [79.01422521024834]
We explore a useful but often neglected methodology for robustness analysis of text generation evaluation metrics.
We design and synthesize a wide range of potential errors and check whether they result in a commensurate drop in the metric scores.
Our experiments reveal interesting insensitivities, biases, or even loopholes in existing metrics.
arXiv Detail & Related papers (2022-12-20T06:24:25Z) - Textual Enhanced Contrastive Learning for Solving Math Word Problems [23.196339273292246]
We propose a Textual Enhanced Contrastive Learning framework, which enforces the models to distinguish semantically similar examples.
We adopt a self-supervised manner strategy to enrich examples with subtle textual variance.
Experimental results show that our method achieves state-of-the-art on both widely used benchmark datasets and also exquisitely designed challenge datasets in English and Chinese.
arXiv Detail & Related papers (2022-11-29T08:44:09Z) - AES Systems Are Both Overstable And Oversensitive: Explaining Why And
Proposing Defenses [66.49753193098356]
We investigate the reason behind the surprising adversarial brittleness of scoring models.
Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models.
We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
arXiv Detail & Related papers (2021-09-24T03:49:38Z) - Toward the Understanding of Deep Text Matching Models for Information
Retrieval [72.72380690535766]
This paper aims at testing whether existing deep text matching methods satisfy some fundamental gradients in information retrieval.
Specifically, four attributions are used in our study, i.e., term frequency constraint, term discrimination constraint, length normalization constraints, and TF-length constraint.
Experimental results on LETOR 4.0 and MS Marco show that all the investigated deep text matching methods satisfy the above constraints with high probabilities in statistics.
arXiv Detail & Related papers (2021-08-16T13:33:15Z) - Competency Problems: On Finding and Removing Artifacts in Language Data [50.09608320112584]
We argue that for complex language understanding tasks, all simple feature correlations are spurious.
We theoretically analyze the difficulty of creating data for competency problems when human bias is taken into account.
arXiv Detail & Related papers (2021-04-17T21:34:10Z) - Learning What Makes a Difference from Counterfactual Examples and
Gradient Supervision [57.14468881854616]
We propose an auxiliary training objective that improves the generalization capabilities of neural networks.
We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task.
Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
arXiv Detail & Related papers (2020-04-20T02:47:49Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.