Positional Artefacts Propagate Through Masked Language Model Embeddings

• URL: http://arxiv.org/abs/2011.04393v3
• Date: Tue, 25 May 2021 01:38:12 GMT
• Title: Positional Artefacts Propagate Through Masked Language Model Embeddings
• Authors: Ziyang Luo, Artur Kulmizev, Xiaoxi Mao
• Abstract summary: We find cases of persistent outlier neurons within BERT and RoBERTa's hidden state vectors. We pre-train the RoBERTa-base models from scratch and find that the outliers disappear without using positional embeddings.
• Score: 16.97378491957158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers. Namely, we find cases of persistent outlier neurons within BERT and RoBERTa's hidden state vectors that consistently bear the smallest or largest values in said vectors. In an attempt to investigate the source of this information, we introduce a neuron-level analysis method, which reveals that the outliers are closely related to information captured by positional embeddings. We also pre-train the RoBERTa-base models from scratch and find that the outliers disappear without using positional embeddings. These outliers, we find, are the major cause of anisotropy of encoders' raw vector spaces, and clipping them leads to increased similarity across vectors. We demonstrate this in practice by showing that clipped vectors can more accurately distinguish word senses, as well as lead to better sentence embeddings when mean pooling. In three supervised tasks, we find that clipping does not affect the performance.

Related papers

• Activation Scaling for Steering and Interpreting Language Models [55.59689963561315]
  We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. We establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa. Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention.
  arXiv  Detail & Related papers  (2024-10-07T12:01:32Z)
• Local Topology Measures of Contextual Language Model Latent Spaces With Applications to Dialogue Term Extraction [4.887047578768969]
  We introduce complexity measures of the local topology of the latent space of a contextual language model. Our work continues a line of research that explores the manifold hypothesis for word embeddings.
  arXiv  Detail & Related papers  (2024-08-07T11:44:32Z)
• Knowledge Trees: Gradient Boosting Decision Trees on Knowledge Neurons as Probing Classifier [0.0]
  Logistic regression on the output representation of the transformer neural network layer is most often used to probing the syntactic properties of the language model. We show that using gradient boosting decision trees at the Knowledge Neuron layer is more advantageous than using logistic regression on the output representations of the transformer layer.
  arXiv  Detail & Related papers  (2023-12-17T15:37:03Z)
• Backpack Language Models [108.65930795825416]
  We present Backpacks, a new neural architecture that marries strong modeling performance with an interface for interpretability and control. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing.
  arXiv  Detail & Related papers  (2023-05-26T09:26:23Z)
• Tsetlin Machine Embedding: Representing Words Using Logical Expressions [10.825099126920028]
  We introduce a Tsetlin Machine-based autoencoder that learns logical clauses self-supervised. The clauses consist of contextual words like "black," "cup," and "hot" to define other words like "coffee" We evaluate our embedding approach on several intrinsic and extrinsic benchmarks, outperforming GLoVe on six classification tasks.
  arXiv  Detail & Related papers  (2023-01-02T15:02:45Z)
• What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary [68.77983831618685]
  We propose to interpret the vector representations produced by dual encoders by projecting them into the model's vocabulary space. We show that the resulting projections contain rich semantic information, and draw connection between them and sparse retrieval.
  arXiv  Detail & Related papers  (2022-12-20T16:03:25Z)
• Extracting Latent Steering Vectors from Pretrained Language Models [14.77762401765532]
  We show that latent vectors can be extracted directly from language model decoders without fine-tuning. Experiments show that there exist steering vectors, which, when added to the hidden states of the language model, generate a target sentence nearly perfectly. We find that distances between steering vectors reflect sentence similarity when evaluated on a textual similarity benchmark.
  arXiv  Detail & Related papers  (2022-05-10T19:04:37Z)
• Leveraging Sparse Linear Layers for Debuggable Deep Networks [86.94586860037049]
  We show how fitting sparse linear models over learned deep feature representations can lead to more debuggable neural networks. The resulting sparse explanations can help to identify spurious correlations, explain misclassifications, and diagnose model biases in vision and language tasks.
  arXiv  Detail & Related papers  (2021-05-11T08:15:25Z)
• Prototypical Representation Learning for Relation Extraction [56.501332067073065]
  This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data. We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations. Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
  arXiv  Detail & Related papers  (2021-03-22T08:11:43Z)
• R$^2$-Net: Relation of Relation Learning Network for Sentence Semantic Matching [58.72111690643359]
  We propose a Relation of Relation Learning Network (R2-Net) for sentence semantic matching. We first employ BERT to encode the input sentences from a global perspective. Then a CNN-based encoder is designed to capture keywords and phrase information from a local perspective. To fully leverage labels for better relation information extraction, we introduce a self-supervised relation of relation classification task.
  arXiv  Detail & Related papers  (2020-12-16T13:11:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.