Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack

• URL: http://arxiv.org/abs/2109.11308v1
• Date: Thu, 23 Sep 2021 11:47:27 GMT
• Title: Breaking BERT: Understanding its Vulnerabilities for Named Entity Recognition through Adversarial Attack
• Authors: Anne Dirkson, Suzan Verberne, Wessel Kraaij
• Abstract summary: Both generic and domain-specific BERT models are widely used for natural language processing (NLP) tasks. In this paper we investigate the vulnerability of BERT models to variation in input data for Named Entity Recognition (NER) through adversarial attack.
• Score: 10.871587311621974
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Both generic and domain-specific BERT models are widely used for natural language processing (NLP) tasks. In this paper we investigate the vulnerability of BERT models to variation in input data for Named Entity Recognition (NER) through adversarial attack. Experimental results show that the original as well as the domain-specific BERT models are highly vulnerable to entity replacement: They can be fooled in 89.2 to 99.4% of the cases to mislabel previously correct entities. BERT models are also vulnerable to variation in the entity context with 20.2 to 45.0% of entities predicted completely wrong and another 29.3 to 53.3% of entities predicted wrong partially. Often a single change is sufficient to fool the model. BERT models seem most vulnerable to changes in the local context of entities. Of the two domain-specific BERT models, the vulnerability of BioBERT is comparable to the original BERT model whereas SciBERT is even more vulnerable. Our results chart the vulnerabilities of BERT models for NER and emphasize the importance of further research into uncovering and reducing these weaknesses.

Related papers

• Memorization of Named Entities in Fine-tuned BERT Models [3.0177210416625115]
  We investigate the extent of named entity memorization in fine-tuned BERT models. We show that a fine-tuned BERT does not generate more named entities specific to the fine-tuning dataset than a BERT model that is pre-trained only.
  arXiv  Detail & Related papers  (2022-12-07T16:20:50Z)
• Entity-aware Transformers for Entity Search [6.107210856380526]
  We show that the entity-enriched BERT model improves effectiveness on entity-oriented queries over a regular BERT model. We also show that the entity information provided by our entity-enriched model particularly helps queries related to less popular entities.
  arXiv  Detail & Related papers  (2022-05-02T11:53:59Z)
• BiBERT: Accurate Fully Binarized BERT [69.35727280997617]
  BiBERT is an accurate fully binarized BERT to eliminate the performance bottlenecks. Our method yields impressive 56.3 times and 31.2 times saving on FLOPs and model size.
  arXiv  Detail & Related papers  (2022-03-12T09:46:13Z)
• BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition [57.2201011783393]
  conditional hidden Markov model (CHMM) CHMM predicts token-wise transition and emission probabilities from the BERT embeddings of the input tokens. It fine-tunes a BERT-based NER model with the labels inferred by CHMM.
  arXiv  Detail & Related papers  (2021-05-26T21:18:48Z)
• Killing Two Birds with One Stone: Stealing Model and Inferring Attribute from BERT-based APIs [26.38350928431939]
  We present an effective model extraction attack, where the adversary can practically steal a BERT-based API. We develop an effective inference attack to expose the sensitive attribute of the training data used by the BERT-based APIs.
  arXiv  Detail & Related papers  (2021-05-23T10:38:23Z)
• Model Extraction and Adversarial Transferability, Your BERT is Vulnerable! [11.425692676973332]
  We show how an adversary can steal a BERT-based API service on multiple benchmark datasets with limited prior knowledge and queries. We also show that the extracted model can lead to highly transferable adversarial attacks against the victim model. Our studies indicate that the potential vulnerabilities of BERT-based API services still hold, even when there is an architectural mismatch between the victim model and the attack model.
  arXiv  Detail & Related papers  (2021-03-18T04:23:21Z)
• BinaryBERT: Pushing the Limit of BERT Quantization [74.65543496761553]
  We propose BinaryBERT, which pushes BERT quantization to the limit with weight binarization. We find that a binary BERT is hard to be trained directly than a ternary counterpart due to its complex and irregular loss landscapes. Empirical results show that BinaryBERT has negligible performance drop compared to the full-precision BERT-base.
  arXiv  Detail & Related papers  (2020-12-31T16:34:54Z)
• TernaryBERT: Distillation-aware Ultra-low Bit BERT [53.06741585060951]
  We propose TernaryBERT, which ternarizes the weights in a fine-tuned BERT model. Experiments on the GLUE benchmark and SQuAD show that our proposed TernaryBERT outperforms the other BERT quantization methods.
  arXiv  Detail & Related papers  (2020-09-27T10:17:28Z)
• ConvBERT: Improving BERT with Span-based Dynamic Convolution [144.25748617961082]
  BERT heavily relies on the global self-attention block and thus suffers large memory footprint and computation cost. We propose a novel span-based dynamic convolution to replace these self-attention heads to directly model local dependencies. The novel convolution heads, together with the rest self-attention heads, form a new mixed attention block that is more efficient at both global and local context learning.
  arXiv  Detail & Related papers  (2020-08-06T07:43:19Z)
• What's in a Name? Are BERT Named Entity Representations just as Good for any other Name? [18.11382921200802]
  We evaluate named entity representations of BERT-based NLP models by investigating their robustness to replacements from the same typed class in the input. We provide a simple method that ensembles predictions from multiple replacements while jointly modeling the uncertainty of type annotations and label predictions. Experiments on three NLP tasks show that our method enhances robustness and increases accuracy on both natural and adversarial datasets.
  arXiv  Detail & Related papers  (2020-07-14T08:14:00Z)
• Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation [84.64004917951547]
  Fine-tuning pre-trained language models like BERT has become an effective way in NLP. In this paper, we improve the fine-tuning of BERT with two effective mechanisms: self-ensemble and self-distillation.
  arXiv  Detail & Related papers  (2020-02-24T16:17:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.