Towards Rigorous Understanding of Neural Networks via Semantics-preserving Transformations

• URL: http://arxiv.org/abs/2301.08013v2
• Date: Fri, 28 Apr 2023 15:00:01 GMT
• Title: Towards Rigorous Understanding of Neural Networks via Semantics-preserving Transformations
• Authors: Maximilian Schl\"uter and Gerrit Nolte and Alnis Murtovi and Bernhard Steffen
• Abstract summary: We present an approach to the precise and global verification and explanation of Rectifier Neural Networks. Key to our approach is the symbolic execution of these networks that allows the construction of semantically equivalent Typed Affine Decision Structures.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we present an algebraic approach to the precise and global verification and explanation of Rectifier Neural Networks, a subclass of Piece-wise Linear Neural Networks (PLNNs), i.e., networks that semantically represent piece-wise affine functions. Key to our approach is the symbolic execution of these networks that allows the construction of semantically equivalent Typed Affine Decision Structures (TADS). Due to their deterministic and sequential nature, TADS can, similarly to decision trees, be considered as white-box models and therefore as precise solutions to the model and outcome explanation problem. TADS are linear algebras which allows one to elegantly compare Rectifier Networks for equivalence or similarity, both with precise diagnostic information in case of failure, and to characterize their classification potential by precisely characterizing the set of inputs that are specifically classified or the set of inputs where two network-based classifiers differ. All phenomena are illustrated along a detailed discussion of a minimal, illustrative example: the continuous XOR function.

Related papers

• Learning local discrete features in explainable-by-design convolutional neural networks [0.0]
  We introduce an explainable-by-design convolutional neural network (CNN) based on the lateral inhibition mechanism. The model consists of the predictor, that is a high-accuracy CNN with residual or dense skip connections. By collecting observations and directly calculating probabilities, we can explain causal relationships between motifs of adjacent levels.
  arXiv  Detail & Related papers  (2024-10-31T18:39:41Z)
• Coding schemes in neural networks learning classification tasks [52.22978725954347]
  We investigate fully-connected, wide neural networks learning classification tasks. We show that the networks acquire strong, data-dependent features. Surprisingly, the nature of the internal representations depends crucially on the neuronal nonlinearity.
  arXiv  Detail & Related papers  (2024-06-24T14:50:05Z)
• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• Combinatorial Convolutional Neural Networks for Words [0.0]
  We argue that the identification of such patterns may be important for certain applications. We argue that the identification of such patterns may be important for certain applications. We present a convolutional neural network for word classification.
  arXiv  Detail & Related papers  (2023-03-28T07:49:06Z)
• Bispectral Neural Networks [1.0323063834827415]
  We present a neural network architecture, Bispectral Neural Networks (BNNs) BNNs are able to simultaneously learn groups, their irreducible representations, and corresponding equivariant and complete-invariant maps.
  arXiv  Detail & Related papers  (2022-09-07T18:34:48Z)
• Interpretable Neural Networks based classifiers for categorical inputs [0.0]
  We introduce a simple way to interpret the output function of a neural network classifier that take as input categorical variables. We show that in these cases each layer of the network, and the logits layer in particular, can be expanded as a sum of terms that account for the contribution to the classification of each input pattern. The analysis of the contributions of each pattern, after an appropriate gauge transformation, is presented in two cases where the effectiveness of the method can be appreciated.
  arXiv  Detail & Related papers  (2021-02-05T14:38:50Z)
• Stability of Algebraic Neural Networks to Small Perturbations [179.55535781816343]
  Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator.
  arXiv  Detail & Related papers  (2020-10-22T09:10:16Z)
• Learning Connectivity of Neural Networks from a Topological Perspective [80.35103711638548]
  We propose a topological perspective to represent a network into a complete graph for analysis. By assigning learnable parameters to the edges which reflect the magnitude of connections, the learning process can be performed in a differentiable manner. This learning process is compatible with existing networks and owns adaptability to larger search spaces and different tasks.
  arXiv  Detail & Related papers  (2020-08-19T04:53:31Z)
• The Projected Belief Network Classfier : both Generative and Discriminative [13.554038901140949]
  The projected belief network (PBN) is a layered generative network with tractable likelihood function. In this paper, a convolutional PBN is constructed that is both fully discriminative and fully generative.
  arXiv  Detail & Related papers  (2020-08-14T16:00:54Z)
• Building powerful and equivariant graph neural networks with structural message-passing [74.93169425144755]
  We propose a powerful and equivariant message-passing framework based on two ideas. First, we propagate a one-hot encoding of the nodes, in addition to the features, in order to learn a local context matrix around each node. Second, we propose methods for the parametrization of the message and update functions that ensure permutation equivariance.
  arXiv  Detail & Related papers  (2020-06-26T17:15:16Z)
• Neural Anisotropy Directions [63.627760598441796]
  We define neural anisotropy directions (NADs) the vectors that encapsulate the directional inductive bias of an architecture. We show that for the CIFAR-10 dataset, NADs characterize the features used by CNNs to discriminate between different classes.
  arXiv  Detail & Related papers  (2020-06-17T08:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.