The Fifth International Verification of Neural Networks Competition (VNN-COMP 2024): Summary and Results

• URL: http://arxiv.org/abs/2412.19985v1
• Date: Sat, 28 Dec 2024 03:07:00 GMT
• Title: The Fifth International Verification of Neural Networks Competition (VNN-COMP 2024): Summary and Results
• Authors: Christopher Brix, Stanley Bak, Taylor T. Johnson, Haoze Wu,
• Abstract summary: This report summarizes the 5th International Verification of Neural Networks Competition (VNN-COMP 2024) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
• Score: 3.9189620165765
• License:
• Abstract: This report summarizes the 5th International Verification of Neural Networks Competition (VNN-COMP 2024), held as a part of the 7th International Symposium on AI Verification (SAIV), that was collocated with the 36th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and specification (VNN-LIB) were defined, tools were evaluated on equal-cost hardware (using an automatic evaluation pipeline based on AWS instances), and tool parameters were chosen by the participants before the final test sets were made public. In the 2024 iteration, 8 teams participated on a diverse set of 12 regular and 8 extended benchmarks. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.

Related papers

• First Competition on Presentation Attack Detection on ID Card [9.872311870613748]
  This paper summarises the Competition on Presentation Attack Detection on ID Cards (PAD-IDCard) held at the 2024 International Joint Conference on Biometrics (IJCB2024) The competition attracted a total of ten registered teams, both from academia and industry. In summary, a team that chose to be "Anonymous" reached the best average ranking results of 74.80%, followed very closely by the "IDVC" team with 77.65%.
  arXiv  Detail & Related papers  (2024-08-31T07:24:19Z)
• The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results [7.3262152011453745]
  This report summarizes the 4th International Verification of Neural Networks Competition (VNN-COMP 2023) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
  arXiv  Detail & Related papers  (2023-12-28T00:46:35Z)
• Perception Test 2023: A Summary of the First Challenge And Outcome [67.0525378209708]
  The First Perception Test challenge was held as a half-day workshop alongside the IEEE/CVF International Conference on Computer Vision (ICCV) 2023. The goal was to benchmarking state-of-the-art video models on the recently proposed Perception Test benchmark. We summarise in this report the task descriptions, metrics, baselines, and results.
  arXiv  Detail & Related papers  (2023-12-20T15:12:27Z)
• SynFacePAD 2023: Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data [51.42380508231581]
  The paper presents a summary of the Competition on Face Presentation Attack Detection Based on Privacy-aware Synthetic Training Data (SynFacePAD 2023) held at the 2023 International Joint Conference on Biometrics (IJCB 2023) The competition aimed to motivate and attract solutions that target detecting face presentation attacks while considering synthetic-based training data motivated by privacy, legal and ethical concerns associated with personal data. The submitted solutions presented innovations and novel approaches that led to outperforming the considered baseline in the investigated benchmarks.
  arXiv  Detail & Related papers  (2023-11-09T13:02:04Z)
• EFaR 2023: Efficient Face Recognition Competition [51.77649060180531]
  The paper presents the summary of the Efficient Face Recognition Competition (EFaR) held at the 2023 International Joint Conference on Biometrics (IJCB 2023) The competition received 17 submissions from 6 different teams. The submitted solutions are ranked based on a weighted score of the achieved verification accuracies on a diverse set of benchmarks, as well as the deployability given by the number of floating-point operations and model size.
  arXiv  Detail & Related papers  (2023-08-08T09:58:22Z)
• First Three Years of the International Verification of Neural Networks Competition (VNN-COMP) [9.02791567988691]
  In the VNN-COMP, participants submit software tools that analyze whether given neural networks satisfy specifications describing their input-output behavior. We summarize the key processes, rules, and results, present trends observed over the last three years, and provide an outlook into possible future developments.
  arXiv  Detail & Related papers  (2023-01-14T04:04:12Z)
• The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results [9.02791567988691]
  This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022) VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.
  arXiv  Detail & Related papers  (2022-12-20T15:58:01Z)
• An ensemble of VisNet, Transformer-M, and pretraining models for molecular property prediction in OGB Large-Scale Challenge @ NeurIPS 2022 [48.109627319222334]
  ViSNet Team achieved the MAE of 0.0723 eV on the test-challenge set, dramatically reducing the error by 39.75% compared with the best method in the last year competition.
  arXiv  Detail & Related papers  (2022-11-23T09:12:17Z)
• The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results [1.4824891788575418]
  This report summarizes the second International Verification of Neural Networks Competition (VNN-COMP 2021) The goal of the competition is to provide an objective comparison of the state-of-the-art methods in neural network verification. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this competition.
  arXiv  Detail & Related papers  (2021-08-31T01:29:56Z)
• SVSNet: An End-to-end Speaker Voice Similarity Assessment Model [61.3813595968834]
  We propose SVSNet, the first end-to-end neural network model to assess the speaker voice similarity between natural speech and synthesized speech. The experimental results on the Voice Conversion Challenge 2018 and 2020 show that SVSNet notably outperforms well-known baseline systems.
  arXiv  Detail & Related papers  (2021-07-20T10:19:46Z)
• Emotion Recognition for In-the-wild Videos [92.01434273996097]
  This paper is a brief introduction to our submission to the seven basic expression classification track of Affective Behavior Analysis in-the-wild Competition held in conjunction with the IEEE International Conference on Automatic Face and Gesture Recognition (FG) 2020. Our method combines Deep Residual Network (ResNet) and Bidirectional Long Short-Term Memory Network (BLSTM), achieving 64.3% accuracy and 43.4% final metric on the validation set.
  arXiv  Detail & Related papers  (2020-02-13T11:29:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.