Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network

• URL: http://arxiv.org/abs/2410.16159v1
• Date: Mon, 21 Oct 2024 16:22:19 GMT
• Title: Metric as Transform: Exploring beyond Affine Transform for Interpretable Neural Network
• Authors: Suman Sapkota,
• Abstract summary: We find dot product neurons with global influence less interpretable as compared to local influence of euclidean distance. We develop an interpretable local dictionary based Neural Networks and use it to understand and reject adversarial examples.
• Score: 2.7195102129095003
• License:
• Abstract: Artificial Neural Networks of varying architectures are generally paired with affine transformation at the core. However, we find dot product neurons with global influence less interpretable as compared to local influence of euclidean distance (as used in Radial Basis Function Network). In this work, we explore the generalization of dot product neurons to $l^p$-norm, metrics, and beyond. We find that metrics as transform performs similarly to affine transform when used in MultiLayer Perceptron or Convolutional Neural Network. Moreover, we explore various properties of Metrics, compare it with Affine, and present multiple cases where metrics seem to provide better interpretability. We develop an interpretable local dictionary based Neural Networks and use it to understand and reject adversarial examples.

Related papers

• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory [64.06519549649495]
  We provide the definition of what we call functionally equivalent features. These features produce equivalent output under certain transformations. We propose an efficient algorithm named Iterative Feature Merging.
  arXiv  Detail & Related papers  (2023-10-10T16:27:12Z)
• Permutation Equivariant Neural Functionals [92.0667671999604]
  This work studies the design of neural networks that can process the weights or gradients of other neural networks. We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order. In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
  arXiv  Detail & Related papers  (2023-02-27T18:52:38Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Similarity and Matching of Neural Network Representations [0.0]
  We employ a toolset -- dubbed Dr. Frankenstein -- to analyse the similarity of representations in deep neural networks. We aim to match the activations on given layers of two trained neural networks by joining them with a stitching layer.
  arXiv  Detail & Related papers  (2021-10-27T17:59:46Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Convolutional Neural Networks Are Not Invariant to Translation, but They Can Learn to Be [0.76146285961466]
  When seeing a new object, humans can immediately recognize it across different retinal locations. It is commonly believed that Convolutional Neural Networks (CNNs) are architecturally invariant to translation. We show how pretraining a network on an environment with the right latent' characteristics can result in the network learning deep perceptual rules.
  arXiv  Detail & Related papers  (2021-10-12T09:51:07Z)
• Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry [55.57953219617467]
  We show that the learnability of a neural network is directly related to its size. We use Betti numbers to measure the topological geometric complexity of input data and the neural network. We perform the experiments on a real-world dataset MNIST and the results verify our analysis and conclusion.
  arXiv  Detail & Related papers  (2021-09-03T11:45:51Z)
• Learning Translation Invariance in CNNs [1.52292571922932]
  We show how, even though CNNs are not 'architecturally invariant' to translation, they can indeed 'learn' to be invariant to translation. We investigated how this pretraining affected the internal network representations. These experiments show how pretraining a network on an environment with the right 'latent' characteristics can result in the network learning deep perceptual rules.
  arXiv  Detail & Related papers  (2020-11-06T09:39:27Z)
• Transformations between deep neural networks [0.0]
  We propose to test, and when possible establish, an equivalence between two different artificial neural networks. We first discuss transformation functions between only the outputs of the two networks. We then consider transformations that take into account outputs (activations) of a number of internal neurons from each network.
  arXiv  Detail & Related papers  (2020-07-10T23:32:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.