The inverse problem for neural networks

• URL: http://arxiv.org/abs/2308.14093v1
• Date: Sun, 27 Aug 2023 12:35:38 GMT
• Title: The inverse problem for neural networks
• Authors: Marcelo Forets and Christian Schilling
• Abstract summary: We study the problem of computing the preimage of a set under a neural network with piecewise-affine activation functions. We show several applications of computing the preimage for analysis and interpretability of neural networks.
• Score: 3.2634122554914002
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the problem of computing the preimage of a set under a neural network with piecewise-affine activation functions. We recall an old result that the preimage of a polyhedral set is again a union of polyhedral sets and can be effectively computed. We show several applications of computing the preimage for analysis and interpretability of neural networks.

Related papers

• Topological obstruction to the training of shallow ReLU neural networks [0.0]
  We study the interplay between the geometry of the loss landscape and the optimization trajectories of simple neural networks. This paper reveals the presence of topological obstruction in the loss landscape of shallow ReLU neural networks trained using gradient flow.
  arXiv  Detail & Related papers  (2024-10-18T19:17:48Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Image segmentation with traveling waves in an exactly solvable recurrent neural network [71.74150501418039]
  We show that a recurrent neural network can effectively divide an image into groups according to a scene's structural characteristics. We present a precise description of the mechanism underlying object segmentation in this network. We then demonstrate a simple algorithm for object segmentation that generalizes across inputs ranging from simple geometric objects in grayscale images to natural images.
  arXiv  Detail & Related papers  (2023-11-28T16:46:44Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• Open- and Closed-Loop Neural Network Verification using Polynomial Zonotopes [6.591194329459251]
  We present a novel approach to efficiently compute tight non-contact activation functions. In particular, we evaluate the input-output relation of each neuron by an approximation. This results in a superior performance compared to other methods.
  arXiv  Detail & Related papers  (2022-07-06T14:39:19Z)
• Optimal Learning Rates of Deep Convolutional Neural Networks: Additive Ridge Functions [19.762318115851617]
  We consider the mean squared error analysis for deep convolutional neural networks. We show that, for additive ridge functions, convolutional neural networks followed by one fully connected layer with ReLU activation functions can reach optimal mini-max rates.
  arXiv  Detail & Related papers  (2022-02-24T14:22:32Z)
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search [65.79109790446257]
  Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior. We propose to search for neural architectures that capture stronger image priors. We search for an improved network by leveraging an existing neural architecture search algorithm.
  arXiv  Detail & Related papers  (2020-08-26T17:59:36Z)
• Neural Sparse Representation for Image Restoration [116.72107034624344]
  Inspired by the robustness and efficiency of sparse coding based image restoration models, we investigate the sparsity of neurons in deep networks. Our method structurally enforces sparsity constraints upon hidden neurons. Experiments show that sparse representation is crucial in deep neural networks for multiple image restoration tasks.
  arXiv  Detail & Related papers  (2020-06-08T05:15:17Z)
• Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation [60.80172153614544]
  Un-trained convolutional neural networks have emerged as highly successful tools for image recovery and restoration. We show that an un-trained convolutional neural network can approximately reconstruct signals and images that are sufficiently structured, from a near minimal number of random measurements.
  arXiv  Detail & Related papers  (2020-05-07T15:57:25Z)
• A function space analysis of finite neural networks with insights from sampling theory [41.07083436560303]
  We show that the function space generated by multi-layer networks with non-expansive activation functions is smooth. Under the assumption that the input is band-limited, we provide novel error bounds. We analyze both deterministic uniform and random sampling showing the advantage of the former.
  arXiv  Detail & Related papers  (2020-04-15T10:25:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.