The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results

• URL: http://arxiv.org/abs/2312.16760v1
• Date: Thu, 28 Dec 2023 00:46:35 GMT
• Title: The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results
• Authors: Christopher Brix, Stanley Bak, Changliu Liu, Taylor T. Johnson
• Abstract summary: This report summarizes the 4th International Verification of Neural Networks Competition (VNN-COMP 2023) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
• Score: 7.3262152011453745
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This report summarizes the 4th International Verification of Neural Networks Competition (VNN-COMP 2023), held as a part of the 6th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), that was collocated with the 35th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and specification (VNN-LIB) were defined, tools were evaluated on equal-cost hardware (using an automatic evaluation pipeline based on AWS instances), and tool parameters were chosen by the participants before the final test sets were made public. In the 2023 iteration, 7 teams participated on a diverse set of 10 scored and 4 unscored benchmarks. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.

Related papers

• The Fifth International Verification of Neural Networks Competition (VNN-COMP 2024): Summary and Results [3.9189620165765]
  This report summarizes the 5th International Verification of Neural Networks Competition (VNN-COMP 2024) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
  arXiv  Detail & Related papers  (2024-12-28T03:07:00Z)
• SynFacePAD 2023: Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data [51.42380508231581]
  The paper presents a summary of the Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data (SynFacePAD 2023) held at the 2023 International Joint Conference on Biometrics (IJCB 2023) The competition aimed to motivate and attract solutions that target detecting face presentation attacks while considering synthetic-based training data motivated by privacy, legal and ethical concerns associated with personal data. The submitted solutions presented innovations and novel approaches that led to outperforming the considered baseline in the investigated benchmarks.
  arXiv  Detail & Related papers  (2023-11-09T13:02:04Z)
• EFaR 2023: Efficient Face Recognition Competition [51.77649060180531]
  The paper presents the summary of the Efficient Face Recognition Competition (EFaR) held at the 2023 International Joint Conference on Biometrics (IJCB 2023) The competition received 17 submissions from 6 different teams. The submitted solutions are ranked based on a weighted score of the achieved verification accuracies on a diverse set of benchmarks, as well as the deployability given by the number of floating-point operations and model size.
  arXiv  Detail & Related papers  (2023-08-08T09:58:22Z)
• First Three Years of the International Verification of Neural Networks Competition (VNN-COMP) [9.02791567988691]
  In the VNN-COMP, participants submit software tools that analyze whether given neural networks satisfy specifications describing their input-output behavior. We summarize the key processes, rules, and results, present trends observed over the last three years, and provide an outlook into possible future developments.
  arXiv  Detail & Related papers  (2023-01-14T04:04:12Z)
• The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results [9.02791567988691]
  This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
  arXiv  Detail & Related papers  (2022-12-20T15:58:01Z)
• An ensemble of VisNet, Transformer-M, and pretraining models for molecular property prediction in OGB Large-Scale Challenge @ NeurIPS 2022 [48.109627319222334]
  ViSNet Team achieved the MAE of 0.0723 eV on the test-challenge set, dramatically reducing the error by 39.75% compared with the best method in the last year competition.
  arXiv  Detail & Related papers  (2022-11-23T09:12:17Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results [1.4824891788575418]
  This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021) The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.
  arXiv  Detail & Related papers  (2021-08-31T01:29:56Z)
• SVSNet: An End-to-end Speaker Voice Similarity Assessment Model [61.3813595968834]
  We propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between natural speech and synthesized speech. The experimental results on the Voice Conversion Challenge 2018 and 2020 show that SVSNet notably outperforms well-known baseline systems.
  arXiv  Detail & Related papers  (2021-07-20T10:19:46Z)
• Machine Learning for Performance Prediction of Channel Bonding in Next-Generation IEEE 802.11 WLANs [1.0486135378491268]
  We present the results gathered from Problem Statement13 (PS-013), organized by Universitat Pompeu Fabra (UPF) The primary goal was predicting the performance of next-generation Wireless Local Area Networks (WLANs) applying Channel Bonding (CB) techniques.
  arXiv  Detail & Related papers  (2021-05-29T05:33:07Z)
• Emotion Recognition for In-the-wild Videos [92.01434273996097]
  This paper is a brief introduction to our submission to the seven basic expression classification track of Affective Behavior Analysis in-the-wild Competition held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020. Our method combines Deep Residual Network (ResNet) and Bidirectional Long Short-Term Memory Network (BLSTM), achieving 64.3% accuracy and 43.4% final metric on the validation set.
  arXiv  Detail & Related papers  (2020-02-13T11:29:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.