Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks

• URL: http://arxiv.org/abs/2503.17630v1
• Date: Sat, 22 Mar 2025 03:19:55 GMT
• Title: Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks
• Authors: Bin Duan, Matthew B. Dwyer, Guowei Yang,
• Abstract summary: ARGUS is a black-box method to generate realistic, diverse, and fault-revealing test inputs.<n>ARGUS excels in generating realistic and diverse adversarial samples relative to the target dataset.<n>Using adversarial samples for model retraining can improve model classification accuracy.
• Score: 23.018072013940245
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep Neural Networks (DNNs) have been widely employed across various domains, including safety-critical systems, necessitating comprehensive testing to ensure their reliability. Although numerous DNN model testing methods have been proposed to generate adversarial samples that are capable of revealing faults, existing methods typically perturb samples in the input space and then mutate these based on feedback from the DNN model. These methods often result in test samples that are not realistic and with low-probability reveal faults. To address these limitations, we propose a black-box DNN test input generation method, ARGUS, to generate realistic, diverse, and fault-revealing test inputs. ARGUS first compresses samples into a continuous latent space and then perturbs the original samples by interpolating these with samples of different classes. Subsequently, we employ a vector quantizer and decoder to reconstruct adversarial samples back into the input space. Additionally, we employ discriminators both in the latent space and in the input space to ensure the realism of the generated samples. Evaluation of ARGUS in comparison with state-of-the-art black-box testing and white-box testing methods, shows that ARGUS excels in generating realistic and diverse adversarial samples relative to the target dataset, and ARGUS successfully perturbs all original samples and achieves up to 4 times higher error rate than the best baseline method. Furthermore, using these adversarial samples for model retraining can improve model classification accuracy.

Related papers

• DOTA: Distributional Test-Time Adaptation of Vision-Language Models [52.98590762456236]
  Training-free test-time dynamic adapter (TDA) is a promising approach to address this issue. We propose a simple yet effective method for DistributiOnal Test-time Adaptation (Dota) Dota continually estimates the distributions of test samples, allowing the model to continually adapt to the deployment environment.
  arXiv  Detail & Related papers  (2024-09-28T15:03:28Z)
• Robust Black-box Testing of Deep Neural Networks using Co-Domain Coverage [18.355332126489756]
  Rigorous testing of machine learning models is necessary for trustworthy deployments. We present a novel black-box approach for generating test-suites for robust testing of deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2024-08-13T09:42:57Z)
• Hypothesis-Driven Deep Learning for Out of Distribution Detection [0.8191518216608217]
  We propose a hypothesis-driven approach to quantify whether a new sample is InD or OoD. We adapt our method to detect an unseen sample of bacteria to a trained deep learning model, and show that it reveals interpretable differences between InD and OoD latent responses.
  arXiv  Detail & Related papers  (2024-03-21T01:06:47Z)
• Breaking Free: How to Hack Safety Guardrails in Black-Box Diffusion Models! [52.0855711767075]
  EvoSeed is an evolutionary strategy-based algorithmic framework for generating photo-realistic natural adversarial samples. We employ CMA-ES to optimize the search for an initial seed vector, which, when processed by the Conditional Diffusion Model, results in the natural adversarial sample misclassified by the Model. Experiments show that generated adversarial images are of high image quality, raising concerns about generating harmful content bypassing safety classifiers.
  arXiv  Detail & Related papers  (2024-02-07T09:39:29Z)
• Continual Test-time Domain Adaptation via Dynamic Sample Selection [38.82346845855512]
  This paper proposes a Dynamic Sample Selection (DSS) method for Continual Test-time Domain Adaptation (CTDA) We apply joint positive and negative learning on both high- and low-quality samples to reduce the risk of using wrong information. Our approach is also evaluated in the 3D point cloud domain, showcasing its versatility and potential for broader applicability.
  arXiv  Detail & Related papers  (2023-10-05T06:35:21Z)
• TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision [70.05605071885914]
  We propose a novel modification of the self-supervised training algorithm SwAV that adds the ability to adapt to single test samples. We show the success of our method on the common benchmark dataset CIFAR10-C.
  arXiv  Detail & Related papers  (2022-05-18T05:43:06Z)
• Normality-Calibrated Autoencoder for Unsupervised Anomaly Detection on Data Contamination [4.547161155818913]
  Normality-Calibrated Autoencoder (NCAE) can boost anomaly detection performance on contaminated datasets. NCAE adversarially generates high confident normal samples from a latent space having low entropy.
  arXiv  Detail & Related papers  (2021-10-28T00:23:01Z)
• DAAIN: Detection of Anomalous and Adversarial Input using Normalizing Flows [52.31831255787147]
  We introduce a novel technique, DAAIN, to detect out-of-distribution (OOD) inputs and adversarial attacks (AA) Our approach monitors the inner workings of a neural network and learns a density estimator of the activation distribution. Our model can be trained on a single GPU making it compute efficient and deployable without requiring specialized accelerators.
  arXiv  Detail & Related papers  (2021-05-30T22:07:13Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• Black-box Adversarial Sample Generation Based on Differential Evolution [18.82850158275813]
  We propose a black-box technique to test the robustness of Deep Neural Networks (DNNs) The technique does not require any knowledge of the structure or weights of the target DNN. Experimental results show that our technique can achieve 100% success in generating adversarial samples to trigger misclassification.
  arXiv  Detail & Related papers  (2020-07-30T08:43:45Z)
• SUOD: Accelerating Large-Scale Unsupervised Heterogeneous Outlier Detection [63.253850875265115]
  Outlier detection (OD) is a key machine learning (ML) task for identifying abnormal objects from general samples. We propose a modular acceleration system, called SUOD, to address it.
  arXiv  Detail & Related papers  (2020-03-11T00:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.