Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decomposition

• URL: http://arxiv.org/abs/2601.07700v2
• Date: Wed, 14 Jan 2026 10:50:16 GMT
• Title: Hidden Monotonicity: Explaining Deep Neural Networks via their DC Decomposition
• Authors: Jakob Paul Zimmermann, Georg Loho,
• Abstract summary: We show that monotonicity can still be used in two ways to boost explainability.<n>First, we use an adaptation of the decomposition of a trained ReLU network into two monotone and convex parts.<n>Second, we exhibit that training a model as the difference between two monotone neural networks results in a system with strong self-explainability properties.
• Score: 1.2103414068933556
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: It has been demonstrated in various contexts that monotonicity leads to better explainability in neural networks. However, not every function can be well approximated by a monotone neural network. We demonstrate that monotonicity can still be used in two ways to boost explainability. First, we use an adaptation of the decomposition of a trained ReLU network into two monotone and convex parts, thereby overcoming numerical obstacles from an inherent blowup of the weights in this procedure. Our proposed saliency methods - SplitCAM and SplitLRP - improve on state of the art results on both VGG16 and Resnet18 networks on ImageNet-S across all Quantus saliency metric categories. Second, we exhibit that training a model as the difference between two monotone neural networks results in a system with strong self-explainability properties.

Related papers

• Combining Causal Models for More Accurate Abstractions of Neural Networks [10.115827125021438]
  Causal abstraction provides a precise notion of when a network implements an algorithm.<n>A typical problem in practical settings is that the algorithm is not an entirely faithful abstraction of the network.<n>We propose a solution where we combine different simple high-level models to produce a more faithful representation of the network.
  arXiv  Detail & Related papers  (2025-03-14T14:14:43Z)
• Learning local discrete features in explainable-by-design convolutional neural networks [0.0]
  We introduce an explainable-by-design convolutional neural network (CNN) based on the lateral inhibition mechanism. The model consists of the predictor, that is a high-accuracy CNN with residual or dense skip connections. By collecting observations and directly calculating probabilities, we can explain causal relationships between motifs of adjacent levels.
  arXiv  Detail & Related papers  (2024-10-31T18:39:41Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Taming Binarized Neural Networks and Mixed-Integer Programs [2.7624021966289596]
  We show that binarized neural networks admit a tame representation. This makes it possible to use the framework of Bolte et al. for implicit differentiation. This approach could also be used for a broader class of mixed-integer programs.
  arXiv  Detail & Related papers  (2023-10-05T21:04:16Z)
• Certified Invertibility in Neural Networks via Mixed-Integer Programming [16.64960701212292]
  Neural networks are known to be vulnerable to adversarial attacks. There may exist large, meaningful perturbations that do not affect the network's decision. We discuss how our findings can be useful for invertibility certification in transformations between neural networks.
  arXiv  Detail & Related papers  (2023-01-27T15:40:38Z)
• Improved Convergence Guarantees for Shallow Neural Networks [91.3755431537592]
  We prove convergence of depth 2 neural networks, trained via gradient descent, to a global minimum. Our model has the following features: regression with quadratic loss function, fully connected feedforward architecture, RelU activations, Gaussian data instances, adversarial labels. They strongly suggest that, at least in our model, the convergence phenomenon extends well beyond the NTK regime''
  arXiv  Detail & Related papers  (2022-12-05T14:47:52Z)
• Constrained Monotonic Neural Networks [0.685316573653194]
  Wider adoption of neural networks in many critical domains such as finance and healthcare is being hindered by the need to explain their predictions. Monotonicity constraint is one of the most requested properties in real-world scenarios. We show it can approximate any continuous monotone function on a compact subset of $mathbbRn$.
  arXiv  Detail & Related papers  (2022-05-24T04:26:10Z)
• Split personalities in Bayesian Neural Networks: the case for full marginalisation [0.0]
  We show that the true posterior distribution of a Bayesian neural network is massively multimodal. It is only by fully marginalising over all posterior modes, using appropriate Bayesian sampling tools, that we can capture the split personalities of the network.
  arXiv  Detail & Related papers  (2022-05-23T09:24:37Z)
• Benign Overfitting in Two-layer Convolutional Neural Networks [90.75603889605043]
  We study the benign overfitting phenomenon in training a two-layer convolutional neural network (CNN) We show that when the signal-to-noise ratio satisfies a certain condition, a two-layer CNN trained by gradient descent can achieve arbitrarily small training and test loss. On the other hand, when this condition does not hold, overfitting becomes harmful and the obtained CNN can only achieve constant level test loss.
  arXiv  Detail & Related papers  (2022-02-14T07:45:51Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)
• Redundant representations help generalization in wide neural networks [71.38860635025907]
  We study the last hidden layer representations of various state-of-the-art convolutional neural networks. We find that if the last hidden representation is wide enough, its neurons tend to split into groups that carry identical information, and differ from each other only by statistically independent noise.
  arXiv  Detail & Related papers  (2021-06-07T10:18:54Z)
• Towards Understanding Hierarchical Learning: Benefits of Neural Representations [160.33479656108926]
  In this work, we demonstrate that intermediate neural representations add more flexibility to neural networks. We show that neural representation can achieve improved sample complexities compared with the raw input. Our results characterize when neural representations are beneficial, and may provide a new perspective on why depth is important in deep learning.
  arXiv  Detail & Related papers  (2020-06-24T02:44:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.