Benchmarking Local Robustness of High-Accuracy Binary Neural Networks for Enhanced Traffic Sign Recognition

• URL: http://arxiv.org/abs/2310.03033v1
• Date: Mon, 25 Sep 2023 01:17:14 GMT
• Title: Benchmarking Local Robustness of High-Accuracy Binary Neural Networks for Enhanced Traffic Sign Recognition
• Authors: Andreea Postovan, M\u{a}d\u{a}lina Era\c{s}cu
• Abstract summary: This paper introduces a set of benchmark problems featuring layers that challenge state-of-the-art verification tools. The difficulty of the verification problem is given by the high number of network parameters.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Traffic signs play a critical role in road safety and traffic management for autonomous driving systems. Accurate traffic sign classification is essential but challenging due to real-world complexities like adversarial examples and occlusions. To address these issues, binary neural networks offer promise in constructing classifiers suitable for resource-constrained devices. In our previous work, we proposed high-accuracy BNN models for traffic sign recognition, focusing on compact size for limited computation and energy resources. To evaluate their local robustness, this paper introduces a set of benchmark problems featuring layers that challenge state-of-the-art verification tools. These layers include binarized convolutions, max pooling, batch normalization, fully connected. The difficulty of the verification problem is given by the high number of network parameters (905k - 1.7 M), of the input dimension (2.7k-12k), and of the number of regions (43) as well by the fact that the neural networks are not sparse. The proposed BNN models and local robustness properties can be checked at https://github.com/ChristopherBrix/vnncomp2023_benchmarks/tree/main/benchmarks/traffic_signs_recogni tion. The results of the 4th International Verification of Neural Networks Competition (VNN-COMP'23) revealed the fact that 4, out of 7, solvers can handle many of our benchmarks randomly selected (minimum is 6, maximum is 36, out of 45). Surprisingly, tools output also wrong results or missing counterexample (ranging from 1 to 4). Currently, our focus lies in exploring the possibility of achieving a greater count of solved instances by extending the allotted time (previously set at 8 minutes). Furthermore, we are intrigued by the reasons behind the erroneous outcomes provided by the tools for certain benchmarks.

Related papers

• Efficient Traffic Classification using HW-NAS: Advanced Analysis and Optimization for Cybersecurity on Resource-Constrained Devices [1.3124513975412255]
  This paper presents a hardware-efficient deep neural network (DNN) optimized through hardware-aware neural architecture search (HW-NAS)<n>It supports the classification of session-level encrypted traffic on resource-constrained Internet of Things (IoT) and edge devices.<n>The optimized model attains an accuracy of 96.59% with just 88.26K parameters, 10.08M FLOPs, and a maximum tensor size of 20.12K.
  arXiv  Detail & Related papers  (2025-06-12T21:37:45Z)
• Entanglement Classification of Arbitrary Three-Qubit States via Artificial Neural Networks [2.715284063484557]
  We design and implement artificial neural networks (ANNs) to detect and classify entanglement for three-qubit systems. The models are trained and validated on a simulated dataset of randomly generated states. Remarkably, we find that feeding only 7 diagonal elements of the density matrix into the ANN results in an accuracy greater than 94% for both tasks.
  arXiv  Detail & Related papers  (2024-11-18T06:50:10Z)
• Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximation [51.235583545740674]
  MaxLin is a robustness verifier for MaxPool-based CNNs with tight linear approximation. We evaluate MaxLin with open-sourced benchmarks, including LeNet and networks trained on the MNIST, CIFAR-10, and Tiny ImageNet datasets.
  arXiv  Detail & Related papers  (2024-06-02T10:33:04Z)
• On Statistical Learning of Branch and Bound for Vehicle Routing Optimization [3.6922704509753084]
  We train neural networks to emulate the decision-making process of the computationally expensive Strong Branching strategy. We find that this approach can match or improve upon the performance of the branch and bound algorithm.
  arXiv  Detail & Related papers  (2023-10-15T23:59:57Z)
• Architecturing Binarized Neural Networks for Traffic Sign Recognition [0.0]
  Binarized neural networks (BNNs) have shown promising results in computationally limited and energy-constrained devices. We propose BNNs architectures which achieve more than $90%$ for the German Traffic Sign Recognition Benchmark (GTSRB) The number of parameters of these architectures varies from 100k to less than 2M.
  arXiv  Detail & Related papers  (2023-03-27T08:46:31Z)
• The #DNN-Verification Problem: Counting Unsafe Inputs for Deep Neural Networks [94.63547069706459]
  #DNN-Verification problem involves counting the number of input configurations of a DNN that result in a violation of a safety property. We propose a novel approach that returns the exact count of violations. We present experimental results on a set of safety-critical benchmarks.
  arXiv  Detail & Related papers  (2023-01-17T18:32:01Z)
• Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks [51.8723187709964]
  We study the OOD generalization of neural algorithmic reasoning tasks. The goal is to learn an algorithm from input-output pairs using deep neural networks.
  arXiv  Detail & Related papers  (2022-11-01T18:33:20Z)
• Object Detection with Spiking Neural Networks on Automotive Event Data [0.0]
  We propose to train spiking neural networks (SNNs) directly on data coming from event cameras to design fast and efficient automotive embedded applications. In this paper, we conducted experiments on two automotive event datasets, establishing new state-of-the-art classification results for spiking neural networks.
  arXiv  Detail & Related papers  (2022-05-09T14:39:47Z)
• A Mixed Integer Programming Approach for Verifying Properties of Binarized Neural Networks [44.44006029119672]
  We propose a mixed integer programming formulation for BNN verification. We demonstrate our approach by verifying properties of BNNs trained on the MNIST dataset and an aircraft collision avoidance controller.
  arXiv  Detail & Related papers  (2022-03-11T01:11:29Z)
• Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks [72.81092567651395]
  Sub-bit Neural Networks (SNNs) are a new type of binary quantization design tailored to compress and accelerate BNNs. SNNs are trained with a kernel-aware optimization framework, which exploits binary quantization in the fine-grained convolutional kernel space. Experiments on visual recognition benchmarks and the hardware deployment on FPGA validate the great potentials of SNNs.
  arXiv  Detail & Related papers  (2021-10-18T11:30:29Z)
• ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked Models [56.21470608621633]
  We propose a time estimation framework to decouple the architectural search from the target hardware. The proposed methodology extracts a set of models from micro- kernel and multi-layer benchmarks and generates a stacked model for mapping and network execution time estimation. We compare estimation accuracy and fidelity of the generated mixed models, statistical models with the roofline model, and a refined roofline model for evaluation.
  arXiv  Detail & Related papers  (2021-05-07T11:39:05Z)
• FATNN: Fast and Accurate Ternary Neural Networks [89.07796377047619]
  Ternary Neural Networks (TNNs) have received much attention due to being potentially orders of magnitude faster in inference, as well as more power efficient, than full-precision counterparts. In this work, we show that, under some mild constraints, computational complexity of the ternary inner product can be reduced by a factor of 2. We elaborately design an implementation-dependent ternary quantization algorithm to mitigate the performance gap.
  arXiv  Detail & Related papers  (2020-08-12T04:26:18Z)
• Widening and Squeezing: Towards Accurate and Efficient QNNs [125.172220129257]
  Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than that of networks with full-precision parameters. Most of existing methods aim to enhance performance of QNNs especially binary neural networks by exploiting more effective training techniques. We address this problem by projecting features in original full-precision networks to high-dimensional quantization features.
  arXiv  Detail & Related papers  (2020-02-03T04:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.