Dynamic Test-Time Augmentation via Differentiable Functions

• URL: http://arxiv.org/abs/2212.04681v3
• Date: Tue, 22 Oct 2024 06:56:00 GMT
• Title: Dynamic Test-Time Augmentation via Differentiable Functions
• Authors: Shohei Enomoto, Monikka Roslianna Busto, Takeharu Eda,
• Abstract summary: DynTTA is an image enhancement method that generates recognition-friendly images without retraining the recognition model. DynTTA is based on differentiable data augmentation techniques and generates a blended image from many augmented images to improve the recognition accuracy under distribution shifts.
• Score: 3.686808512438363
• License:
• Abstract: Distribution shifts, which often occur in the real world, degrade the accuracy of deep learning systems, and thus improving robustness to distribution shifts is essential for practical applications. To improve robustness, we study an image enhancement method that generates recognition-friendly images without retraining the recognition model. We propose a novel image enhancement method, DynTTA, which is based on differentiable data augmentation techniques and generates a blended image from many augmented images to improve the recognition accuracy under distribution shifts. In addition to standard data augmentations, DynTTA also incorporates deep neural network-based image transformation, further improving the robustness. Because DynTTA is composed of differentiable functions, it can be directly trained with the classification loss of the recognition model. In experiments with widely used image recognition datasets using various classification models, DynTTA improves the robustness with almost no reduction in classification accuracy for clean images, thus outperforming the existing methods. Furthermore, the results show that robustness is significantly improved by estimating the training-time augmentations for distribution-shifted datasets using DynTTA and retraining the recognition model with the estimated augmentations. DynTTA is a promising approach for applications that require both clean accuracy and robustness. Our code is available at \url{https://github.com/s-enmt/DynTTA}.

Related papers

• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• Adversarial Masking Contrastive Learning for vein recognition [10.886119051977785]
  Vein recognition has received increasing attention due to its high security and privacy. Deep neural networks such as Convolutional neural networks (CNN) and Transformers have been introduced for vein recognition. Despite the recent advances, existing solutions for finger-vein feature extraction are still not optimal due to scarce training image samples.
  arXiv  Detail & Related papers  (2024-01-16T03:09:45Z)
• Image edge enhancement for effective image classification [7.470763273994321]
  We propose an edge enhancement-based method to enhance both accuracy and training speed of neural networks. Our approach involves extracting high frequency features, such as edges, from images within the available dataset and fusing them with the original images.
  arXiv  Detail & Related papers  (2024-01-13T10:01:34Z)
• X-Transfer: A Transfer Learning-Based Framework for GAN-Generated Fake Image Detection [33.31312811230408]
  misuse of GANs for generating deceptive images, such as face replacement, raises significant security concerns. This paper introduces a novel GAN-generated image detection algorithm called X-Transfer. It enhances transfer learning by utilizing two neural networks that employ interleaved parallel gradient transmission.
  arXiv  Detail & Related papers  (2023-10-07T01:23:49Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• Effective Data Augmentation With Diffusion Models [65.09758931804478]
  We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.
  arXiv  Detail & Related papers  (2023-02-07T20:42:28Z)
• Does Data Augmentation Benefit from Split BatchNorms [29.134017115737507]
  State-of-the-art data augmentation strongly distorts training images, leading to a disparity between examples seen during training and inference. We propose an auxiliary BatchNorm for the potentially out-of-distribution, strongly augmented images. We find that this method significantly improves the performance of common image classification benchmarks such as CIFAR-10, CIFAR-100, and ImageNet.
  arXiv  Detail & Related papers  (2020-10-15T15:00:43Z)
• Encoding Robustness to Image Style via Adversarial Feature Perturbations [72.81911076841408]
  We adapt adversarial training by directly perturbing feature statistics, rather than image pixels, to produce robust models. Our proposed method, Adversarial Batch Normalization (AdvBN), is a single network layer that generates worst-case feature perturbations during training.
  arXiv  Detail & Related papers  (2020-09-18T17:52:34Z)
• Learning to Learn Parameterized Classification Networks for Scalable Input Images [76.44375136492827]
  Convolutional Neural Networks (CNNs) do not have a predictable recognition behavior with respect to the input resolution change. We employ meta learners to generate convolutional weights of main networks for various input scales. We further utilize knowledge distillation on the fly over model predictions based on different input resolutions.
  arXiv  Detail & Related papers  (2020-07-13T04:27:25Z)
• Image Augmentations for GAN Training [57.65145659417266]
  We provide insights and guidelines on how to augment images for both vanilla GANs and GANs with regularizations. Surprisingly, we find that vanilla GANs attain generation quality on par with recent state-of-the-art results.
  arXiv  Detail & Related papers  (2020-06-04T00:16:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.