AttentionViz: A Global View of Transformer Attention

• URL: http://arxiv.org/abs/2305.03210v2
• Date: Wed, 9 Aug 2023 06:24:55 GMT
• Title: AttentionViz: A Global View of Transformer Attention
• Authors: Catherine Yeh, Yida Chen, Aoyu Wu, Cynthia Chen, Fernanda Vi\'egas, Martin Wattenberg
• Abstract summary: We present a new visualization technique designed to help researchers understand the self-attention mechanism in transformers. The main idea behind our method is to visualize a joint embedding of the query and key vectors used by transformer models to compute attention. We create an interactive visualization tool, AttentionViz, based on these joint query-key embeddings.
• Score: 60.82904477362676
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformer models are revolutionizing machine learning, but their inner workings remain mysterious. In this work, we present a new visualization technique designed to help researchers understand the self-attention mechanism in transformers that allows these models to learn rich, contextual relationships between elements of a sequence. The main idea behind our method is to visualize a joint embedding of the query and key vectors used by transformer models to compute attention. Unlike previous attention visualization techniques, our approach enables the analysis of global patterns across multiple input sequences. We create an interactive visualization tool, AttentionViz (demo: http://attentionviz.com), based on these joint query-key embeddings, and use it to study attention mechanisms in both language and vision transformers. We demonstrate the utility of our approach in improving model understanding and offering new insights about query-key interactions through several application scenarios and expert feedback.

Related papers

• VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers [45.42482446288144]
  Recent advances in interpretability suggest we can project weights and hidden states of transformer-based language models to their vocabulary. We investigate LM attention heads and memory values, the vectors the models dynamically create and recall while processing a given input. We create a tool to visualize a forward pass of Generative Pre-trained Transformers (GPTs) as an interactive flow graph.
  arXiv  Detail & Related papers  (2023-05-22T19:04:56Z)
• Masked World Models for Visual Control [90.13638482124567]
  We introduce a visual model-based RL framework that decouples visual representation learning and dynamics learning. We demonstrate that our approach achieves state-of-the-art performance on a variety of visual robotic tasks.
  arXiv  Detail & Related papers  (2022-06-28T18:42:27Z)
• VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers [47.581265194864585]
  Internal mechanisms of vision and multimodal transformers remain largely opaque. With the success of these transformers, it is increasingly critical to understand their inner workings. We propose VL-InterpreT, which provides novel interactive visualizations for interpreting the attentions and hidden representations in multimodal transformers.
  arXiv  Detail & Related papers  (2022-03-30T05:25:35Z)
• Visualizing and Understanding Patch Interactions in Vision Transformer [96.70401478061076]
  Vision Transformer (ViT) has become a leading tool in various computer vision tasks. We propose a novel explainable visualization approach to analyze and interpret the crucial attention interactions among patches for vision transformer.
  arXiv  Detail & Related papers  (2022-03-11T13:48:11Z)
• Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers [78.26411729589526]
  We propose the first method to explain prediction by any Transformer-based architecture. Our method is superior to all existing methods which are adapted from single modality explainability.
  arXiv  Detail & Related papers  (2021-03-29T15:03:11Z)
• Dodrio: Exploring Transformer Models with Interactive Visualization [10.603327364971559]
  Dodrio is an open-source interactive visualization tool to help NLP researchers and practitioners analyze attention mechanisms in transformer-based models with linguistic knowledge. To facilitate the visual comparison of attention weights and linguistic knowledge, Dodrio applies different graph visualization techniques to represent attention weights with longer input text.
  arXiv  Detail & Related papers  (2021-03-26T17:39:37Z)
• SparseBERT: Rethinking the Importance Analysis in Self-attention [107.68072039537311]
  Transformer-based models are popular for natural language processing (NLP) tasks due to its powerful capacity. Attention map visualization of a pre-trained model is one direct method for understanding self-attention mechanism. We propose a Differentiable Attention Mask (DAM) algorithm, which can be also applied in guidance of SparseBERT design.
  arXiv  Detail & Related papers  (2021-02-25T14:13:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.