Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognition

• URL: http://arxiv.org/abs/2203.10693v1
• Date: Mon, 21 Mar 2022 01:21:12 GMT
• Title: Leveraging Expert Guided Adversarial Augmentation For Improving Generalization in Named Entity Recognition
• Authors: Aaron Reich, Jiaao Chen, Aastha Agrawal, Yanzhe Zhang and Diyi Yang
• Abstract summary: Named Entity Recognition (NER) systems often demonstrate great performance on in-distribution data, but perform poorly on examples drawn from a shifted distribution. We propose leveraging expert-guideds to change the entity tokens and their surrounding contexts thereby altering their entity types as adversarial attacks. We found that state-of-the-art NER systems trained on CoNLL 2003 training data drop performance dramatically on our challenging set.
• Score: 50.85774164546487
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Named Entity Recognition (NER) systems often demonstrate great performance on in-distribution data, but perform poorly on examples drawn from a shifted distribution. One way to evaluate the generalization ability of NER models is to use adversarial examples, on which the specific variations associated with named entities are rarely considered. To this end, we propose leveraging expert-guided heuristics to change the entity tokens and their surrounding contexts thereby altering their entity types as adversarial attacks. Using expert-guided heuristics, we augmented the CoNLL 2003 test set and manually annotated it to construct a high-quality challenging set. We found that state-of-the-art NER systems trained on CoNLL 2003 training data drop performance dramatically on our challenging set. By training on adversarial augmented training examples and using mixup for regularization, we were able to significantly improve the performance on the challenging set as well as improve out-of-domain generalization which we evaluated by using OntoNotes data. We have publicly released our dataset and code at https://github.com/GT-SALT/Guided-Adversarial-Augmentation.

Related papers

• Boosting Adversarial Training via Fisher-Rao Norm-based Regularization [9.975998980413301]
  We propose a novel regularization framework, called Logit-Oriented Adversarial Training (LOAT), which can mitigate the trade-off between robustness and accuracy. Our experiments demonstrate that the proposed regularization strategy can boost the performance of the prevalent adversarial training algorithms.
  arXiv  Detail & Related papers  (2024-03-26T09:22:37Z)
• Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances [55.37242480995541]
  We propose to denoise noisy NER data with guidance from a small set of clean instances. Along with the main NER model we train a discriminator model and use its outputs to recalibrate the sample weights. Results on public crowdsourcing and distant supervision datasets show that the proposed method can consistently improve performance with a small guidance set.
  arXiv  Detail & Related papers  (2023-10-25T17:23:37Z)
• Consistency Regularization for Generalizable Source-free Domain Adaptation [62.654883736925456]
  Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset. Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets. We propose a consistency regularization framework to develop a more generalizable SFDA method.
  arXiv  Detail & Related papers  (2023-08-03T07:45:53Z)
• Self-supervised debiasing using low rank regularization [59.84695042540525]
  Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. We propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
  arXiv  Detail & Related papers  (2022-10-11T08:26:19Z)
• Generalizable Person Re-Identification via Self-Supervised Batch Norm Test-Time Adaption [63.7424680360004]
  Batch Norm Test-time Adaption (BNTA) is a novel re-id framework that applies the self-supervised strategy to update BN parameters adaptively. BNTA explores the domain-aware information within unlabeled target data before inference, and accordingly modulates the feature distribution normalized by BN to adapt to the target domain.
  arXiv  Detail & Related papers  (2022-03-01T18:46:32Z)
• Feature Diversity Learning with Sample Dropout for Unsupervised Domain Adaptive Person Re-identification [0.0]
  This paper proposes a new approach to learn the feature representation with better generalization ability through limiting noisy pseudo labels. We put forward a brand-new method referred as to Feature Diversity Learning (FDL) under the classic mutual-teaching architecture. Experimental results show that our proposed FDL-SD achieves the state-of-the-art performance on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2022-01-25T10:10:48Z)
• Unsupervised Learning of Debiased Representations with Pseudo-Attributes [85.5691102676175]
  We propose a simple but effective debiasing technique in an unsupervised manner. We perform clustering on the feature embedding space and identify pseudoattributes by taking advantage of the clustering results. We then employ a novel cluster-based reweighting scheme for learning debiased representation.
  arXiv  Detail & Related papers  (2021-08-06T05:20:46Z)
• Practical Assessment of Generalization Performance Robustness for Deep Networks via Contrastive Examples [36.50563671470897]
  Training images with data transformations have been suggested as contrastive examples to complement the testing set for generalization performance evaluation of deep neural networks (DNNs) In this work, we propose a practical framework ContRE that uses Contrastive examples for DNN geneRalization performance Estimation.
  arXiv  Detail & Related papers  (2021-06-20T08:46:01Z)
• Regularizing Models via Pointwise Mutual Information for Named Entity Recognition [17.767466724342064]
  We propose a Pointwise Mutual Information (PMI) to enhance generalization ability while outperforming an in-domain performance. Our approach enables to debias highly correlated word and labels in the benchmark datasets. For long-named and complex-structure entities, our method can predict these entities through debiasing on conjunction or special characters.
  arXiv  Detail & Related papers  (2021-04-15T05:47:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.