Automated Testing of AI Models

• URL: http://arxiv.org/abs/2110.03320v1
• Date: Thu, 7 Oct 2021 10:30:18 GMT
• Title: Automated Testing of AI Models
• Authors: Swagatam Haldar, Deepak Vijaykeerthy, Diptikalyan Saha
• Abstract summary: We extend the capability of the AITEST tool to include the testing techniques for Image and Speech-to-text models. These novel extensions make AITEST a comprehensive framework for testing AI models.
• Score: 3.0616624345970975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The last decade has seen tremendous progress in AI technology and applications. With such widespread adoption, ensuring the reliability of the AI models is crucial. In past, we took the first step of creating a testing framework called AITEST for metamorphic properties such as fairness, robustness properties for tabular, time-series, and text classification models. In this paper, we extend the capability of the AITEST tool to include the testing techniques for Image and Speech-to-text models along with interpretability testing for tabular models. These novel extensions make AITEST a comprehensive framework for testing AI models.

Related papers

• SONAR: A Synthetic AI-Audio Detection Framework and Benchmark [59.09338266364506]
  SONAR is a synthetic AI-Audio Detection Framework and Benchmark. It aims to provide a comprehensive evaluation for distinguishing cutting-edge AI-synthesized auditory content. It is the first framework to uniformly benchmark AI-audio detection across both traditional and foundation model-based deepfake detection systems.
  arXiv  Detail & Related papers  (2024-10-06T01:03:42Z)
• Automatic Generation of Behavioral Test Cases For Natural Language Processing Using Clustering and Prompting [6.938766764201549]
  This paper introduces an automated approach to develop test cases by exploiting the power of large language models and statistical techniques. We analyze the behavioral test profiles across four different classification algorithms and discuss the limitations and strengths of those models.
  arXiv  Detail & Related papers  (2024-07-31T21:12:21Z)
• XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approach [2.0209172586699173]
  This paper introduces a novel XAI-integrated Visual Quality Inspection framework. Our framework incorporates XAI and the Large Vision Language Model to deliver human-centered interpretability. This approach paves the way for the broader adoption of reliable and interpretable AI tools in critical industrial applications.
  arXiv  Detail & Related papers  (2024-07-16T14:30:24Z)
• AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving [68.73885845181242]
  We propose an Automatic Data Engine (AIDE) that automatically identifies issues, efficiently curates data, improves the model through auto-labeling, and verifies the model through generation of diverse scenarios. We further establish a benchmark for open-world detection on AV datasets to comprehensively evaluate various learning paradigms, demonstrating our method's superior performance at a reduced cost.
  arXiv  Detail & Related papers  (2024-03-26T04:27:56Z)
• IoTGeM: Generalizable Models for Behaviour-Based IoT Attack Detection [3.3772986620114387]
  We present an approach for modelling IoT network attacks that focuses on generalizability, yet also leads to better detection and performance. First, we present an improved rolling window approach for feature extraction, and introduce a multi-step feature selection process that reduces overfitting. Second, we build and test models using isolated train and test datasets, thereby avoiding common data leaks. Third, we rigorously evaluate our methodology using a diverse portfolio of machine learning models, evaluation metrics and datasets.
  arXiv  Detail & Related papers  (2023-10-17T21:46:43Z)
• AI-Generated Images as Data Source: The Dawn of Synthetic Era [61.879821573066216]
  generative AI has unlocked the potential to create synthetic images that closely resemble real-world photographs. This paper explores the innovative concept of harnessing these AI-generated images as new data sources. In contrast to real data, AI-generated data exhibit remarkable advantages, including unmatched abundance and scalability.
  arXiv  Detail & Related papers  (2023-10-03T06:55:19Z)
• RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model [93.8067369210696]
  Text-to-image generation (TTI) refers to the usage of models that could process text input and generate high fidelity images based on text descriptions. Diffusion models are one prominent type of generative model used for the generation of images through the systematic introduction of noises with repeating steps. In the era of large models, scaling up model size and the integration with large language models have further improved the performance of TTI models.
  arXiv  Detail & Related papers  (2023-09-02T03:27:20Z)
• On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Training [109.9218185711916]
  Aspect-based sentiment analysis (ABSA) aims at automatically inferring the specific sentiment polarities toward certain aspects of products or services behind social media texts or reviews. We propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training.
  arXiv  Detail & Related papers  (2023-04-19T11:07:43Z)
• Data Synthesis for Testing Black-Box Machine Learning Models [2.3800397174740984]
  The increasing usage of machine learning models raises the question of the reliability of these models. In this paper, we provide a framework for automated test data synthesis to test black-box ML/DL models.
  arXiv  Detail & Related papers  (2021-11-03T12:00:30Z)
• Testing Framework for Black-box AI Models [1.916485402892365]
  In this paper, we present an end-to-end generic framework for testing AI Models. Our tool has been used for testing industrial AI models and was very effective to uncover issues.
  arXiv  Detail & Related papers  (2021-02-11T18:15:23Z)
• Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems [64.4896118325552]
  We evaluate the current state-of-the-art AES models using a model adversarial evaluation scheme and associated metrics. We find that AES models are highly overstable. Even heavy modifications(as much as 25%) with content unrelated to the topic of the questions do not decrease the score produced by the models.
  arXiv  Detail & Related papers  (2020-07-14T03:49:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.