PREMAP: A Unifying PREiMage APproximation Framework for Neural Networks

• URL: http://arxiv.org/abs/2408.09262v1
• Date: Sat, 17 Aug 2024 17:24:47 GMT
• Title: PREMAP: A Unifying PREiMage APproximation Framework for Neural Networks
• Authors: Xiyue Zhang, Benjie Wang, Marta Kwiatkowska, Huan Zhang,
• Abstract summary: We present a framework for preimage abstraction that produces under- and over-approximations of any polyhedral output set. We evaluate our method on a range of tasks, demonstrating significant improvement in efficiency and scalability to high-input-dimensional image classification tasks.
• Score: 30.701422594374456
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most methods for neural network verification focus on bounding the image, i.e., set of outputs for a given input set. This can be used to, for example, check the robustness of neural network predictions to bounded perturbations of an input. However, verifying properties concerning the preimage, i.e., the set of inputs satisfying an output property, requires abstractions in the input space. We present a general framework for preimage abstraction that produces under- and over-approximations of any polyhedral output set. Our framework employs cheap parameterised linear relaxations of the neural network, together with an anytime refinement procedure that iteratively partitions the input region by splitting on input features and neurons. The effectiveness of our approach relies on carefully designed heuristics and optimization objectives to achieve rapid improvements in the approximation volume. We evaluate our method on a range of tasks, demonstrating significant improvement in efficiency and scalability to high-input-dimensional image classification tasks compared to state-of-the-art techniques. Further, we showcase the application to quantitative verification and robustness analysis, presenting a sound and complete algorithm for the former and providing sound quantitative results for the latter.

Related papers

• Forward-Forward Algorithm for Hyperspectral Image Classification: A Preliminary Study [0.0]
  Forward-forward algorithm (FFA) computes local goodness functions to optimize network parameters. This study investigates the application of FFA for hyperspectral image classification.
  arXiv  Detail & Related papers  (2023-07-01T05:39:28Z)
• Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
  Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
  arXiv  Detail & Related papers  (2023-06-13T01:18:16Z)
• Provable Preimage Under-Approximation for Neural Networks (Full Version) [27.519993407376862]
  We propose an efficient anytime algorithm for generating symbolic under-approximations of the preimage of any polyhedron output set for neural networks. Empirically, we validate the efficacy of our method across a range of domains, including a high-dimensional MNIST classification task. We present a sound and complete algorithm for the former, which exploits our disjoint union of polytopes representation to provide formal guarantees.
  arXiv  Detail & Related papers  (2023-05-05T16:55:27Z)
• OReX: Object Reconstruction from Planar Cross-sections Using Neural Fields [10.862993171454685]
  OReX is a method for 3D shape reconstruction from slices alone, featuring a Neural Field gradients as the prior. A modest neural network is trained on the input planes to return an inside/outside estimate for a given 3D coordinate, yielding a powerful prior that induces smoothness and self-similarities. We offer an iterative estimation architecture and a hierarchical input sampling scheme that encourage coarse-to-fine training, allowing the training process to focus on high frequencies at later stages.
  arXiv  Detail & Related papers  (2022-11-23T11:44:35Z)
• Deep Equilibrium Assisted Block Sparse Coding of Inter-dependent Signals: Application to Hyperspectral Imaging [71.57324258813675]
  A dataset of inter-dependent signals is defined as a matrix whose columns demonstrate strong dependencies. A neural network is employed to act as structure prior and reveal the underlying signal interdependencies. Deep unrolling and Deep equilibrium based algorithms are developed, forming highly interpretable and concise deep-learning-based architectures.
  arXiv  Detail & Related papers  (2022-03-29T21:00:39Z)
• Bayesian Attention Belief Networks [59.183311769616466]
  Attention-based neural networks have achieved state-of-the-art results on a wide range of tasks. This paper introduces Bayesian attention belief networks, which construct a decoder network by modeling unnormalized attention weights. We show that our method outperforms deterministic attention and state-of-the-art attention in accuracy, uncertainty estimation, generalization across domains, and adversarial attacks.
  arXiv  Detail & Related papers  (2021-06-09T17:46:22Z)
• Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection [4.683939045230724]
  Current object detectors are difficult to strike a balance between high accuracy and fast inference speed. We present an efficient and robust fully convolutional neural network model to perform robotic grasping pose estimation. The network is an order of magnitude smaller than other excellent algorithms.
  arXiv  Detail & Related papers  (2021-01-25T16:36:53Z)
• WoodFisher: Efficient Second-Order Approximation for Neural Network Compression [35.45199662813043]
  We develop a method to compute a faithful and efficient estimate of the inverse Hessian. Our main application is to neural network compression. We show how our method can be extended to take into account first-order information.
  arXiv  Detail & Related papers  (2020-04-29T17:14:23Z)
• RAIN: A Simple Approach for Robust and Accurate Image Classification Networks [156.09526491791772]
  It has been shown that the majority of existing adversarial defense methods achieve robustness at the cost of sacrificing prediction accuracy. This paper proposes a novel preprocessing framework, which we term Robust and Accurate Image classificatioN(RAIN) RAIN applies randomization over inputs to break the ties between the model forward prediction path and the backward gradient path, thus improving the model robustness. We conduct extensive experiments on the STL10 and ImageNet datasets to verify the effectiveness of RAIN against various types of adversarial attacks.
  arXiv  Detail & Related papers  (2020-04-24T02:03:56Z)
• Parallelization Techniques for Verifying Neural Networks [52.917845265248744]
  We introduce an algorithm based on the verification problem in an iterative manner and explore two partitioning strategies. We also introduce a highly parallelizable pre-processing algorithm that uses the neuron activation phases to simplify the neural network verification problems.
  arXiv  Detail & Related papers  (2020-04-17T20:21:47Z)
• Resolution Adaptive Networks for Efficient Inference [53.04907454606711]
  We propose a novel Resolution Adaptive Network (RANet), which is inspired by the intuition that low-resolution representations are sufficient for classifying "easy" inputs. In RANet, the input images are first routed to a lightweight sub-network that efficiently extracts low-resolution representations. High-resolution paths in the network maintain the capability to recognize the "hard" samples.
  arXiv  Detail & Related papers  (2020-03-16T16:54:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.