Improving Attention-Based Interpretability of Text Classification Transformers

• URL: http://arxiv.org/abs/2209.10876v1
• Date: Thu, 22 Sep 2022 09:19:22 GMT
• Title: Improving Attention-Based Interpretability of Text Classification Transformers
• Authors: Nikolaos Mylonas, Ioannis Mollas, Grigorios Tsoumakas
• Abstract summary: We study the effectiveness of attention-based interpretability techniques for transformers in text classification. We show that, with proper setup, attention may be used in such tasks with results comparable to state-of-the-art techniques.
• Score: 7.027858121801477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformers are widely used in NLP, where they consistently achieve state-of-the-art performance. This is due to their attention-based architecture, which allows them to model rich linguistic relations between words. However, transformers are difficult to interpret. Being able to provide reasoning for its decisions is an important property for a model in domains where human lives are affected, such as hate speech detection and biomedicine. With transformers finding wide use in these fields, the need for interpretability techniques tailored to them arises. The effectiveness of attention-based interpretability techniques for transformers in text classification is studied in this work. Despite concerns about attention-based interpretations in the literature, we show that, with proper setup, attention may be used in such tasks with results comparable to state-of-the-art techniques, while also being faster and friendlier to the environment. We validate our claims with a series of experiments that employ a new feature importance metric.

Related papers

• Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task [40.85615657802704]
  This paper introduces the contextual counting task, a novel toy problem aimed at enhancing our understanding of Transformers. We present theoretical and empirical analysis using both causal and non-causal Transformer architectures.
  arXiv  Detail & Related papers  (2024-05-30T20:52:23Z)
• Explaining Text Similarity in Transformer Models [52.571158418102584]
  Recent advances in explainable AI have made it possible to mitigate limitations by leveraging improved explanations for Transformers. We use BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, to investigate which feature interactions drive similarity in NLP models. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights.
  arXiv  Detail & Related papers  (2024-05-10T17:11:31Z)
• Emotion Detection with Transformers: A Comparative Study [0.0]
  We train and evaluate several pre-trained transformer models, on the Emotion dataset using different variants of transformers. Our analysis reveals that commonly applied techniques like removing punctuation and stop words can hinder model performance.
  arXiv  Detail & Related papers  (2024-03-18T23:22:50Z)
• Adventures of Trustworthy Vision-Language Models: A Survey [54.76511683427566]
  This paper conducts a thorough examination of vision-language transformers, employing three fundamental principles of responsible AI: Bias, Robustness, and Interpretability. The primary objective of this paper is to delve into the intricacies and complexities associated with the practical use of transformers, with the overarching goal of advancing our comprehension of how to enhance their reliability and accountability.
  arXiv  Detail & Related papers  (2023-12-07T11:31:20Z)
• Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining [0.5735035463793008]
  This paper studies the behaviour of the cutting-edge Transformer-based language models on opinion mining. Our comparative study shows leads and paves the way for production engineers regarding the approach to focus on.
  arXiv  Detail & Related papers  (2023-08-07T01:10:50Z)
• A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks [60.38369406877899]
  Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. Our survey encompasses the identification of the top five application domains for transformer-based models.
  arXiv  Detail & Related papers  (2023-06-11T23:13:51Z)
• How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding [56.222097640468306]
  We provide mechanistic understanding of how transformers learn "semantic structure" We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
  arXiv  Detail & Related papers  (2023-03-07T21:42:17Z)
• Probing Speech Emotion Recognition Transformers for Linguistic Knowledge [7.81884995637243]
  We investigate the extent in which linguistic information is exploited during speech emotion recognition fine-tuning. We synthesise prosodically neutral speech utterances while varying the sentiment of the text. Valence predictions of the transformer model are very reactive to positive and negative sentiment content, as well as negations, but not to intensifiers or reducers.
  arXiv  Detail & Related papers  (2022-04-01T12:47:45Z)
• Incorporating Residual and Normalization Layers into Analysis of Masked Language Models [29.828669678974983]
  We extend the scope of the analysis of Transformers from solely the attention patterns to the whole attention block. Our analysis of Transformer-based masked language models shows that the token-to-token interaction performed via attention has less impact on the intermediate representations than previously assumed.
  arXiv  Detail & Related papers  (2021-09-15T08:32:20Z)
• Transformer Interpretability Beyond Attention Visualization [87.96102461221415]
  Self-attention techniques, and specifically Transformers, are dominating the field of text processing. In this work, we propose a novel way to compute relevancy for Transformer networks.
  arXiv  Detail & Related papers  (2020-12-17T18:56:33Z)
• Pretrained Transformers for Text Ranking: BERT and Beyond [53.83210899683987]
  This survey provides an overview of text ranking with neural network architectures known as transformers. The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in natural language processing.
  arXiv  Detail & Related papers  (2020-10-13T15:20:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.