OMENN: One Matrix to Explain Neural Networks

• URL: http://arxiv.org/abs/2412.02399v1
• Date: Tue, 03 Dec 2024 11:49:01 GMT
• Title: OMENN: One Matrix to Explain Neural Networks
• Authors: Adam Wróbel, Mikołaj Janusz, Bartosz Zieliński, Dawid Rymarczyk,
• Abstract summary: One Matrix to Explain Neural Networks (OMENN) is a novel post-hoc method that represents a neural network as a single, interpretable matrix for each specific input. We present a theoretical analysis of OMENN based on dynamic linearity property and validate its effectiveness with extensive tests on two XAI benchmarks.
• Score: 2.397390211883228
• License:
• Abstract: Deep Learning (DL) models are often black boxes, making their decision-making processes difficult to interpret. This lack of transparency has driven advancements in eXplainable Artificial Intelligence (XAI), a field dedicated to clarifying the reasoning behind DL model predictions. Among these, attribution-based methods such as LRP and GradCAM are widely used, though they rely on approximations that can be imprecise. To address these limitations, we introduce One Matrix to Explain Neural Networks (OMENN), a novel post-hoc method that represents a neural network as a single, interpretable matrix for each specific input. This matrix is constructed through a series of linear transformations that represent the processing of the input by each successive layer in the neural network. As a result, OMENN provides locally precise, attribution-based explanations of the input across various modern models, including ViTs and CNNs. We present a theoretical analysis of OMENN based on dynamic linearity property and validate its effectiveness with extensive tests on two XAI benchmarks, demonstrating that OMENN is competitive with state-of-the-art methods.

Related papers

• GINN-KAN: Interpretability pipelining with applications in Physics Informed Neural Networks [5.2969467015867915]
  We introduce the concept of interpretability pipelineing, to incorporate multiple interpretability techniques to outperform each individual technique. We evaluate two recent models selected for their potential to incorporate interpretability into standard neural network architectures. We introduce a novel interpretable neural network GINN-KAN that synthesizes the advantages of both models.
  arXiv  Detail & Related papers  (2024-08-27T04:57:53Z)
• Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations [54.17275171325324]
  We present a counterexample to the Linear Representation Hypothesis (LRH) When trained to repeat an input token sequence, neural networks learn to represent the token at each position with a particular order of magnitude, rather than a direction. These findings strongly indicate that interpretability research should not be confined to the LRH.
  arXiv  Detail & Related papers  (2024-08-20T15:04:37Z)
• Mechanistic Neural Networks for Scientific Machine Learning [58.99592521721158]
  We present Mechanistic Neural Networks, a neural network design for machine learning applications in the sciences. It incorporates a new Mechanistic Block in standard architectures to explicitly learn governing differential equations as representations. Central to our approach is a novel Relaxed Linear Programming solver (NeuRLP) inspired by a technique that reduces solving linear ODEs to solving linear programs.
  arXiv  Detail & Related papers  (2024-02-20T15:23:24Z)
• Instance-wise Linearization of Neural Network for Model Interpretation [13.583425552511704]
  The challenge can dive into the non-linear behavior of the neural network. For a neural network model, the non-linear behavior is often caused by non-linear activation units of a model. We propose an instance-wise linearization approach to reformulates the forward computation process of a neural network prediction.
  arXiv  Detail & Related papers  (2023-10-25T02:07:39Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient Inference [20.404864470321897]
  We introduce NeuralMatrix, which elastically transforms the computations of entire deep neural network (DNN) models into linear matrix operations. Experiments with both CNN and transformer-based models demonstrate the potential of NeuralMatrix to accurately and efficiently execute a wide range of DNN models. This level of efficiency is usually only attainable with the accelerator designed for a specific neural network.
  arXiv  Detail & Related papers  (2023-05-23T12:03:51Z)
• The Contextual Lasso: Sparse Linear Models via Deep Neural Networks [5.607237982617641]
  We develop a new statistical estimator that fits a sparse linear model to the explanatory features such that the sparsity pattern and coefficients vary as a function of the contextual features. An extensive suite of experiments on real and synthetic data suggests that the learned models, which remain highly transparent, can be sparser than the regular lasso.
  arXiv  Detail & Related papers  (2023-02-02T05:00:29Z)
• Connecting Weighted Automata, Tensor Networks and Recurrent Neural Networks through Spectral Learning [58.14930566993063]
  We present connections between three models used in different research fields: weighted finite automata(WFA) from formal languages and linguistics, recurrent neural networks used in machine learning, and tensor networks. We introduce the first provable learning algorithm for linear 2-RNN defined over sequences of continuous vectors input.
  arXiv  Detail & Related papers  (2020-10-19T15:28:00Z)
• Binarized Graph Neural Network [65.20589262811677]
  We develop a binarized graph neural network to learn the binary representations of the nodes with binary network parameters. Our proposed method can be seamlessly integrated into the existing GNN-based embedding approaches. Experiments indicate that the proposed binarized graph neural network, namely BGN, is orders of magnitude more efficient in terms of both time and space.
  arXiv  Detail & Related papers  (2020-04-19T09:43:14Z)
• Adaptive Explainable Neural Networks (AxNNs) [8.949704905866888]
  We develop a new framework called Adaptive Explainable Neural Networks (AxNN) for achieving the dual goals of good predictive performance and model interpretability. For predictive performance, we build a structured neural network made up of ensembles of generalized additive model networks and additive index models. For interpretability, we show how to decompose the results of AxNN into main effects and higher-order interaction effects.
  arXiv  Detail & Related papers  (2020-04-05T23:40:57Z)
• Matrix Smoothing: A Regularization for DNN with Transition Matrix under Noisy Labels [54.585681272543056]
  Training deep neural networks (DNNs) in the presence of noisy labels is an important and challenging task. Recent probabilistic methods directly apply transition matrix to DNN, neglect DNN's susceptibility to overfitting. We propose a novel method, in which a smoothed transition matrix is used for updating DNN, to restrict the overfitting.
  arXiv  Detail & Related papers  (2020-03-26T13:49:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.