VisBERT: Hidden-State Visualizations for Transformers

• URL: http://arxiv.org/abs/2011.04507v1
• Date: Mon, 9 Nov 2020 15:37:43 GMT
• Title: VisBERT: Hidden-State Visualizations for Transformers
• Authors: Betty van Aken, Benjamin Winter, Alexander L\"oser, Felix A. Gers
• Abstract summary: We present VisBERT, a tool for visualizing the contextual token representations within BERT for the task of (multi-hop) Question Answering. VisBERT enables users to get insights about the model's internal state and to explore its inference steps or potential shortcomings.
• Score: 66.86452388524886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Explainability and interpretability are two important concepts, the absence of which can and should impede the application of well-performing neural networks to real-world problems. At the same time, they are difficult to incorporate into the large, black-box models that achieve state-of-the-art results in a multitude of NLP tasks. Bidirectional Encoder Representations from Transformers (BERT) is one such black-box model. It has become a staple architecture to solve many different NLP tasks and has inspired a number of related Transformer models. Understanding how these models draw conclusions is crucial for both their improvement and application. We contribute to this challenge by presenting VisBERT, a tool for visualizing the contextual token representations within BERT for the task of (multi-hop) Question Answering. Instead of analyzing attention weights, we focus on the hidden states resulting from each encoder block within the BERT model. This way we can observe how the semantic representations are transformed throughout the layers of the model. VisBERT enables users to get insights about the model's internal state and to explore its inference steps or potential shortcomings. The tool allows us to identify distinct phases in BERT's transformations that are similar to a traditional NLP pipeline and offer insights during failed predictions.

Related papers

• The Mechanics of Conceptual Interpretation in GPT Models: Interpretative Insights [10.777646083061395]
  We introduce concept editing'', an innovative variation of knowledge editing that uncovers conceptualisation mechanisms within large language models. We analyse the Multi-Layer Perceptron (MLP), Multi-Head Attention (MHA), and hidden state components of transformer models. Our work highlights the complex, layered nature of semantic processing in LLMs and the challenges of isolating and modifying specific concepts within these models.
  arXiv  Detail & Related papers  (2024-08-05T18:50:08Z)
• Towards Understanding How Transformers Learn In-context Through a Representation Learning Lens [9.590540796223715]
  In this paper, we attempt to explore the in-context learning process in Transformers through a lens of representation learning. The ICL inference process of the attention layer aligns with the training procedure of its dual model, generating token representation predictions. We extend our theoretical conclusions to more complicated scenarios, including one Transformer layer and multiple attention layers.
  arXiv  Detail & Related papers  (2023-10-20T01:55:34Z)
• All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations [69.3461199976959]
  We propose a model based on invertible neural networks, BERT-INN, to learn the Bijection Hypothesis. We show the advantage of BERT-INN both theoretically and through extensive experiments.
  arXiv  Detail & Related papers  (2023-05-23T22:30:43Z)
• On Robustness of Finetuned Transformer-based NLP Models [11.063628128069736]
  We characterize changes between pretrained and finetuned language model representations across layers using two metrics: CKA and STIR. GPT-2 representations are more robust than BERT and T5 across multiple types of input perturbations. This study provides valuable insights into perturbation-specific weaknesses of popular Transformer-based models.
  arXiv  Detail & Related papers  (2023-05-23T18:25:18Z)
• VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers [45.42482446288144]
  Recent advances in interpretability suggest we can project weights and hidden states of transformer-based language models to their vocabulary. We investigate LM attention heads and memory values, the vectors the models dynamically create and recall while processing a given input. We create a tool to visualize a forward pass of Generative Pre-trained Transformers (GPTs) as an interactive flow graph.
  arXiv  Detail & Related papers  (2023-05-22T19:04:56Z)
• Holistically Explainable Vision Transformers [136.27303006772294]
  We propose B-cos transformers, which inherently provide holistic explanations for their decisions. Specifically, we formulate each model component - such as the multi-layer perceptrons, attention layers, and the tokenisation module - to be dynamic linear. We apply our proposed design to Vision Transformers (ViTs) and show that the resulting models, dubbed Bcos-ViTs, are highly interpretable and perform competitively to baseline ViTs.
  arXiv  Detail & Related papers  (2023-01-20T16:45:34Z)
• What Makes for Good Tokenizers in Vision Transformer? [62.44987486771936]
  transformers are capable of extracting their pairwise relationships using self-attention. What makes for a good tokenizer has not been well understood in computer vision. Modulation across Tokens (MoTo) incorporates inter-token modeling capability through normalization. Regularization objective TokenProp is embraced in the standard training regime.
  arXiv  Detail & Related papers  (2022-12-21T15:51:43Z)
• VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers [47.581265194864585]
  Internal mechanisms of vision and multimodal transformers remain largely opaque. With the success of these transformers, it is increasingly critical to understand their inner workings. We propose VL-InterpreT, which provides novel interactive visualizations for interpreting the attentions and hidden representations in multimodal transformers.
  arXiv  Detail & Related papers  (2022-03-30T05:25:35Z)
• Visual Saliency Transformer [127.33678448761599]
  We develop a novel unified model based on a pure transformer, Visual Saliency Transformer (VST), for both RGB and RGB-D salient object detection (SOD) It takes image patches as inputs and leverages the transformer to propagate global contexts among image patches. Experimental results show that our model outperforms existing state-of-the-art results on both RGB and RGB-D SOD benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-25T08:24:06Z)
• A Recurrent Vision-and-Language BERT for Navigation [54.059606864535304]
  We propose a recurrent BERT model that is time-aware for use in vision-and-language navigation. Our model can replace more complex encoder-decoder models to achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2020-11-26T00:23:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.