Unveiling Transformer Perception by Exploring Input Manifolds

• URL: http://arxiv.org/abs/2410.06019v1
• Date: Tue, 8 Oct 2024 13:20:31 GMT
• Title: Unveiling Transformer Perception by Exploring Input Manifolds
• Authors: Alessandro Benfenati, Alfio Ferrara, Alessio Marta, Davide Riva, Elisabetta Rocchetti,
• Abstract summary: This paper introduces a general method for the exploration of equivalence classes in the input space of Transformer models. The proposed approach is based on sound mathematical theory which describes the internal layers of a Transformer architecture as sequential deformations of the input manifold.
• Score: 41.364418162255184
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: This paper introduces a general method for the exploration of equivalence classes in the input space of Transformer models. The proposed approach is based on sound mathematical theory which describes the internal layers of a Transformer architecture as sequential deformations of the input manifold. Using eigendecomposition of the pullback of the distance metric defined on the output space through the Jacobian of the model, we are able to reconstruct equivalence classes in the input space and navigate across them. We illustrate how this method can be used as a powerful tool for investigating how a Transformer sees the input space, facilitating local and task-agnostic explainability in Computer Vision and Natural Language Processing tasks.

Related papers

• A prototype-based model for set classification [2.0564549686015594]
  A common way to represent a set of vectors is to model them as linear subspaces. We present a prototype-based approach for learning on the manifold formed from such linear subspaces, the Grassmann manifold.
  arXiv  Detail & Related papers  (2024-08-25T04:29:18Z)
• How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations [98.7450564309923]
  This paper takes initial steps on understanding in-context learning (ICL) in more complex scenarios, by studying learning with representations. We construct synthetic in-context learning problems with a compositional structure, where the label depends on the input through a possibly complex but fixed representation function. We show theoretically the existence of transformers that approximately implement such algorithms with mild depth and size.
  arXiv  Detail & Related papers  (2023-10-16T17:40:49Z)
• Representational Strengths and Limitations of Transformers [33.659870765923884]
  We establish both positive and negative results on the representation power of attention layers. We show the necessity and role of a large embedding dimension in a transformer. We also present natural variants that can be efficiently solved by attention layers.
  arXiv  Detail & Related papers  (2023-06-05T14:05:04Z)
• Learning Expressive Prompting With Residuals for Vision Transformers [11.342913284654706]
  We present Expressive Prompts with Residuals (EXPRES) which modifies the prompt learning paradigm specifically for effective adaptation of vision transformers (ViT) We apply EXPRES for image classification, few shot learning, and semantic segmentation, and show our method is capable of achieving state of the art prompt tuning on 3/3 categories of the VTAB benchmark.
  arXiv  Detail & Related papers  (2023-03-27T20:47:01Z)
• Formal Semantic Geometry over Transformer-based Variational AutoEncoder [10.209508287003155]
  We frame the sentence semantics as the composition of textitsemantic role - word content features and propose the formal semantic geometry. We propose a new probing algorithm to guide the movement of sentence vectors over such geometry. Experimental results reveal that the formal semantic geometry can potentially deliver better control and interpretation to sentence generation.
  arXiv  Detail & Related papers  (2022-10-12T14:20:33Z)
• Analyzing Transformers in Embedding Space [59.434807802802105]
  We present a theoretical analysis where all parameters of a trained Transformer are interpreted by projecting them into the embedding space. We show that parameters of both pretrained and fine-tuned models can be interpreted in embedding space. Our findings open the door to interpretation methods that, at least in part, abstract away from model specifics and operate in the embedding space only.
  arXiv  Detail & Related papers  (2022-09-06T14:36:57Z)
• SOInter: A Novel Deep Energy Based Interpretation Method for Explaining Structured Output Models [6.752231769293388]
  We propose a novel interpretation technique to explain the behavior of structured output models. We focus on one of the outputs as the target and try to find the most important features utilized by the structured model to decide on the target in each locality of the input space.
  arXiv  Detail & Related papers  (2022-02-20T21:57:07Z)
• XAI for Transformers: Better Explanations through Conservative Propagation [60.67748036747221]
  We show that the gradient in a Transformer reflects the function only locally, and thus fails to reliably identify the contribution of input features to the prediction. Our proposal can be seen as a proper extension of the well-established LRP method to Transformers.
  arXiv  Detail & Related papers  (2022-02-15T10:47:11Z)
• Transforming Feature Space to Interpret Machine Learning Models [91.62936410696409]
  This contribution proposes a novel approach that interprets machine-learning models through the lens of feature space transformations. It can be used to enhance unconditional as well as conditional post-hoc diagnostic tools. A case study on remote-sensing landcover classification with 46 features is used to demonstrate the potential of the proposed approach.
  arXiv  Detail & Related papers  (2021-04-09T10:48:11Z)
• Quantitative Understanding of VAE as a Non-linearly Scaled Isometric Embedding [52.48298164494608]
  Variational autoencoder (VAE) estimates the posterior parameters of latent variables corresponding to each input data. This paper provides a quantitative understanding of VAE property through the differential geometric and information-theoretic interpretations of VAE.
  arXiv  Detail & Related papers  (2020-07-30T02:37:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.