Image Augmentation as Test Generation for Deep Learning-Based Image Retrieval Systems

• URL: http://arxiv.org/abs/2608.27502v1
• Date: Thu, 27 Aug 2026 03:40:00 GMT
• Title: Image Augmentation as Test Generation for Deep Learning-Based Image Retrieval Systems
• Abstract summary: This paper reviews augmentation and generation techniques and evaluates them as test generators for embedding-based image retrieval systems.<n>Weather simulation and SaSPA are the image augmentation/generation techniques that produce the highest embedding uncertainty and failure rates.<n>GAN-based augmentation techniques are among the lowest in realism, indicating the presence of synthetic artifacts and perceptual inconsistencies.
• Score: 5.527117132133789
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ensuring the reliability of deep learning-based image retrieval systems is a software engineering challenge. This paper presents a dual contribution: (1) a literature review of augmentation and generation techniques which resulted in the identification of 50 techniques which we organized into a ten-category taxonomy, and (2) a large-scale empirical study that evaluates these techniques as test generators for embedding-based image retrieval systems. Augmented images are embedded using Amazon Titan and OpenCLIP, and evaluated across four analytical dimensions: (1) embedding-space similarity, (2) embedding uncertainty measured via four estimators, (3) semantic realism scored by LLaVA, and (4) retrieval failure rate. Experiments are performed on three datasets: CIFAR-10, ImageNet-1K, and a dataset from an industrial partner (March Networks). Across all evaluated datasets and embedding models, and under the single severity level tested for each technique, weather simulation and SaSPA are the image augmentation/generation techniques that produce the highest embedding uncertainty and failure rates while maintaining a favorable balance between performance stability, visual realism, and augmentation effectiveness. The results we discuss are configuration-specific and may shift under milder or stronger perturbation settings. In contrast, GAN-based augmentation techniques are among the lowest in realism, indicating the presence of synthetic artifacts and perceptual inconsistencies that reduce their suitability to produce realistic test inputs. Overall, our findings provide practical guidelines for selecting augmentation techniques that maximize test diversity while preserving realistic image characteristics, thereby enabling the construction of comprehensive and effective test suites for image retrieval systems while reducing the cost of manual data labeling through the use of metamorphic testing.

Related papers

• Data Safety: Synthetic Data Quality Analysis Using CIFAKE Dataset [0.4784604186682396]
  This study systematically analyzes the differences between two types of synthetic images created by different generation methods and real images from three perspectives.<n>It experimentally verifies how synthetic data should be utilized by considering realistic data mixing scenarios.
  arXiv  Detail & Related papers  (2026-07-13T21:18:15Z)
• Evaluating Synthetic Images as Effective Substitutes for Experimental Data in Surface Roughness Classification [1.697618076775192]
  We show that augmenting authentic datasets with generative images yields test accuracies comparable to those obtained using exclusively experimental images.<n>Our results indicate that generative AI can substantially improve data efficiency and reliability in materials-image classification.
  arXiv  Detail & Related papers  (2026-03-26T06:00:17Z)
• FeatDistill: A Feature Distillation Enhanced Multi-Expert Ensemble Framework for Robust AI-generated Image Detection [14.180599240051238]
  FeatDistill is an AI-generated image detection framework that integrates feature distillation with a multi-expert ensemble.<n>It was developed for the NTIRE Challenge on Robust AI-Generated Image Detection in the Wild.
  arXiv  Detail & Related papers  (2026-03-23T12:55:31Z)
• Implicit Neural Representation-Based Continuous Single Image Super Resolution: An Empirical Study [50.15623093332659]
  Implicit neural representation (INR) has become the standard approach for arbitrary-scale image super-resolution (ASSR)<n>We compare existing techniques across diverse settings and present aggregated performance results on multiple image quality metrics.<n>We examine a new loss function that penalizes intensity variations while preserving edges, textures, and finer details during training.
  arXiv  Detail & Related papers  (2026-01-25T07:09:20Z)
• Bridging the Gap Between Ideal and Real-world Evaluation: Benchmarking AI-Generated Image Detection in Challenging Scenarios [54.07895223545793]
  This paper introduces the Real-World Robustness dataset (RRDataset) for comprehensive evaluation of detection models across three dimensions.<n>RRDataset includes high-quality images from seven major scenarios.<n>We benchmarked 17 detectors and 10 vision-language models (VLMs) on RRDataset and conducted a large-scale human study.
  arXiv  Detail & Related papers  (2025-09-11T06:15:52Z)
• Learning Unpaired Image Dehazing with Physics-based Rehazy Generation [50.37414006427923]
  Overfitting to synthetic training pairs remains a critical challenge in image dehazing.<n>We propose a novel training strategy for unpaired image dehazing, termed Rehazy, to improve both dehazing performance and training stability.
  arXiv  Detail & Related papers  (2025-06-15T12:12:28Z)
• So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection [75.79507634008631]
  We introduce So-Fake-Set, a social media-oriented dataset with over 2 million high-quality images, diverse generative sources, and imagery synthesized using 35 state-of-the-art generative models.<n>We present So-Fake-R1, an advanced vision-language framework that employs reinforcement learning for highly accurate forgery detection, precise localization, and explainable inference through interpretable visual rationales.
  arXiv  Detail & Related papers  (2025-05-24T11:53:35Z)
• Is Artificial Intelligence Generated Image Detection a Solved Problem? [16.9828631287303]
  AIGIBench is a benchmark designed to rigorously evaluate the robustness and generalization capabilities of state-of-the-art AIGI detectors.<n>It simulates real-world challenges through four core tasks: multi-source generalization, robustness to image degradation, sensitivity to data augmentation, and impact of test-time pre-processing.<n>It includes 23 diverse fake image subsets that span both advanced and widely adopted image generation techniques, along with real-world samples collected from social media and AI art platforms.
  arXiv  Detail & Related papers  (2025-05-18T10:00:39Z)
• Enhancing Content Representation for AR Image Quality Assessment Using Knowledge Distillation [3.020452010930984]
  This paper presents a deep learning-based objective metric designed specifically for assessing image quality for Augmented Reality scenarios.<n>It entails four key steps, (1) fine-tuning a self-supervised pre-trained vision transformer to extract prominent features from reference images, (2) quantifying distortions by computing shift representations, (3) employing cross-attention-based decoders to capture perceptual quality features, and (4) integrating regularization techniques and label smoothing to address the overfitting problem.
  arXiv  Detail & Related papers  (2024-12-08T17:25:30Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models.<n>In this paper, we investigate how detection performance varies across model backbones, types, and datasets.<n>We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization [56.95046107046027]
  We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for scene coordinate regression. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain.
  arXiv  Detail & Related papers  (2023-10-10T20:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.