The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results

• URL: http://arxiv.org/abs/2212.10376v1
• Date: Tue, 20 Dec 2022 15:58:01 GMT
• Title: The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results
• Authors: Mark Niklas M\"uller, Christopher Brix, Stanley Bak, Changliu Liu, Taylor T. Johnson
• Abstract summary: This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
• Score: 9.02791567988691
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022), held as a part of the 5th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), which was collocated with the 34th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and specification (VNN-LIB) were defined, tools were evaluated on equal-cost hardware (using an automatic evaluation pipeline based on AWS instances), and tool parameters were chosen by the participants before the final test sets were made public. In the 2022 iteration, 11 teams participated on a diverse set of 12 scored benchmarks. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.

Related papers

• The Fifth International Verification of Neural Networks Competition (VNN-COMP 2024): Summary and Results [3.9189620165765]
  This report summarizes the 5th International Verification of Neural Networks Competition (VNN-COMP 2024) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
  arXiv  Detail & Related papers  (2024-12-28T03:07:00Z)
• The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results [7.3262152011453745]
  This report summarizes the 4th International Verification of Neural Networks Competition (VNN-COMP 2023) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
  arXiv  Detail & Related papers  (2023-12-28T00:46:35Z)
• Severity classification of ground-glass opacity via 2-D convolutional neural network and lung CT scans: a 3-day exploration [0.0]
  Ground-glass opacity is a hallmark of numerous lung diseases, including patients with COVID19 and pneumonia, pulmonary fibrosis, and tuberculosis. This note presents experimental results of a proof-of-concept framework that got implemented and tested over three days as driven by the third challenge entitled "COVID-19 Competition" As part of the challenge requirement, the source code produced during the course of this exercise is posted at https://github.com/lisatwyw/cov19.
  arXiv  Detail & Related papers  (2023-03-23T22:35:37Z)
• First Three Years of the International Verification of Neural Networks Competition (VNN-COMP) [9.02791567988691]
  In the VNN-COMP, participants submit software tools that analyze whether given neural networks satisfy specifications describing their input-output behavior. We summarize the key processes, rules, and results, present trends observed over the last three years, and provide an outlook into possible future developments.
  arXiv  Detail & Related papers  (2023-01-14T04:04:12Z)
• Batch-Ensemble Stochastic Neural Networks for Out-of-Distribution Detection [55.028065567756066]
  Out-of-distribution (OOD) detection has recently received much attention from the machine learning community due to its importance in deploying machine learning models in real-world applications. In this paper we propose an uncertainty quantification approach by modelling the distribution of features. We incorporate an efficient ensemble mechanism, namely batch-ensemble, to construct the batch-ensemble neural networks (BE-SNNs) and overcome the feature collapse problem. We show that BE-SNNs yield superior performance on several OOD benchmarks, such as the Two-Moons dataset, the FashionMNIST vs MNIST dataset, FashionM
  arXiv  Detail & Related papers  (2022-06-26T16:00:22Z)
• GCoNet+: A Stronger Group Collaborative Co-Salient Object Detector [156.43671738038657]
  We present a novel end-to-end group collaborative learning network, termed GCoNet+. GCoNet+ can effectively and efficiently identify co-salient objects in natural scenes.
  arXiv  Detail & Related papers  (2022-05-30T23:49:19Z)
• The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results [1.4824891788575418]
  This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021) The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.
  arXiv  Detail & Related papers  (2021-08-31T01:29:56Z)
• SVSNet: An End-to-end Speaker Voice Similarity Assessment Model [61.3813595968834]
  We propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between natural speech and synthesized speech. The experimental results on the Voice Conversion Challenge 2018 and 2020 show that SVSNet notably outperforms well-known baseline systems.
  arXiv  Detail & Related papers  (2021-07-20T10:19:46Z)
• Analysing Affective Behavior in the second ABAW2 Competition [70.86998050535944]
  The Affective Behavior Analysis in-the-wild (ABAW2) 2021 Competition is the second -- following the first very successful ABAW Competition held in conjunction with IEEE FG 2020- Competition that aims at automatically analyzing affect.
  arXiv  Detail & Related papers  (2021-06-14T11:30:19Z)
• A Two-Stage Approach to Device-Robust Acoustic Scene Classification [63.98724740606457]
  Two-stage system based on fully convolutional neural networks (CNNs) is proposed to improve device robustness. Our results show that the proposed ASC system attains a state-of-the-art accuracy on the development set. Neural saliency analysis with class activation mapping gives new insights on the patterns learnt by our models.
  arXiv  Detail & Related papers  (2020-11-03T03:27:18Z)
• Benchmarking Graph Neural Networks [75.42159546060509]
  Graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. For any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. GitHub repository has reached 1,800 stars and 339 forks, which demonstrates the utility of the proposed open-source framework.
  arXiv  Detail & Related papers  (2020-03-02T15:58:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.