Causal Explanations from the Geometric Properties of ReLU Neural Networks

• URL: http://arxiv.org/abs/2605.10396v1
• Date: Mon, 11 May 2026 11:39:15 GMT
• Title: Causal Explanations from the Geometric Properties of ReLU Neural Networks
• Abstract summary: We aim to interpret the decision making processes of neural networks and autonomous agents.<n>Recent work on understanding the geometry of ReLU neural networks shows that a ReLU network corresponds to a piecewise linear function divided into regions.<n>We show that this geometric representation can be used to generate causal explanations for the network's behaviour.
• Score: 0.2519906683279152
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Neural networks have proved an effective means of learning control policies for autonomous systems, but these learned policies are difficult to understand due to the black-box nature of neural networks. This lack of interpretability makes safety assurance for such autonomous systems challenging. The fields of eXplainable Artificial Intelligence (XAI) and eXplainable Reinforcement Learning (XRL) aim to interpret the decision making processes of neural networks and autonomous agents, respectively. In particular, work on causal explanations aims to provide "why" and "why not" explanations for why a model made a given decision. However, most of the work on explainability to date utilises a distilled version of the original model. While this distilled policy is interpretable, it necessarily degrades in performance significantly when compared to the original model, and is not guaranteed to be an accurate reflection of the decision making processes in the original model and as such cannot be used to guarantee its safety. Recent work on understanding the geometry of ReLU neural networks shows that a ReLU network corresponds to a piecewise linear function divided into regions defined by an n-dimensional convex polytope. Through this lens, a neural network can be understood as dividing the input space into distinct regions which apply a single linear function for each output neuron. We show that this geometric representation can be used to generate causal explanations for the network's behaviour similar to previous work, but which extracts rules directly from the geometry of Neural Networks with the ReLU activation function, and is therefore an accurate reflection of the network's behaviour.

Related papers

• xDNN(ASP): Explanation Generation System for Deep Neural Networks powered by Answer Set Programming [1.1936305787902912]
  xDNN(ASP) is an explanation generation system for deep neural networks that provides global explanations.<n>We show that xDNN(ASP) maintains a high-level of accuracy in the prediction task.
  arXiv  Detail & Related papers  (2026-01-07T12:08:00Z)
• Efficiently Transforming Neural Networks into Decision Trees: A Path to Ground Truth Explanations with RENTT [3.4872074978471788]
  We present an exact equivalent decision tree representation of neural networks in a manner that is both runtime and memory efficient.<n>We also provide a method to calculate the ground truth feature importance for neural networks via the equivalent decision trees.
  arXiv  Detail & Related papers  (2025-11-12T13:07:48Z)
• Concept-Guided Interpretability via Neural Chunking [64.6429903327095]
  We show that neural networks exhibit patterns in their raw population activity that mirror regularities in the training data.<n>We propose three methods to extract recurring chunks on a neural population level.<n>Our work points to a new direction for interpretability, one that harnesses both cognitive principles and the structure of naturalistic data.
  arXiv  Detail & Related papers  (2025-05-16T13:49:43Z)
• neuralGAM: An R Package for Fitting Generalized Additive Neural Networks [0.0]
  The neuralGAM package implements a Neural Network topology based on Generalized Additive Models.<n>The package provides a flexible framework for training Generalized Additive Neural Networks.<n>We illustrate the use of the neuralGAM package in both synthetic and real data examples.
  arXiv  Detail & Related papers  (2025-05-13T14:30:01Z)
• Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations [54.17275171325324]
  We present a counterexample to the Linear Representation Hypothesis (LRH) When trained to repeat an input token sequence, neural networks learn to represent the token at each position with a particular order of magnitude, rather than a direction. These findings strongly indicate that interpretability research should not be confined to the LRH.
  arXiv  Detail & Related papers  (2024-08-20T15:04:37Z)
• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Manipulating Feature Visualizations with Gradient Slingshots [53.94925202421929]
  Feature Visualization (FV) is a widely used technique for interpreting the concepts learned by Deep Neural Networks (DNNs)<n>We introduce a novel method, Gradient Slingshots, that enables manipulation of FV without modifying the model architecture or significantly degrading its performance.
  arXiv  Detail & Related papers  (2024-01-11T18:57:17Z)
• A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions [1.5236380958983644]
  We introduce 'infomorphic' neural networks to perform tasks from supervised, unsupervised and memory learning.<n>By leveraging the interpretable nature of the PID framework, infomorphic networks represent a valuable tool to advance our understanding of the intricate structure of local learning.
  arXiv  Detail & Related papers  (2023-06-03T16:34:25Z)
• Points of non-linearity of functions generated by random neural networks [0.0]
  We consider functions from the real numbers to the real numbers, output by a neural network with 1 hidden activation layer, arbitrary width, and ReLU activation function. We compute the expected distribution of the points of non-linearity.
  arXiv  Detail & Related papers  (2023-04-19T17:40:19Z)
• Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network [23.465930256410722]
  Nonlocal kernel network (NKN) is resolution independent, characterized by deep neural networks. NKN is capable of handling a variety of tasks such as learning governing equations and classifying images.
  arXiv  Detail & Related papers  (2022-01-06T19:19:35Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.