Covariance-aware Feature Alignment with Pre-computed Source Statistics for Test-time Adaptation to Multiple Image Corruptions

• URL: http://arxiv.org/abs/2204.13263v2
• Date: Thu, 29 Jun 2023 07:08:00 GMT
• Title: Covariance-aware Feature Alignment with Pre-computed Source Statistics for Test-time Adaptation to Multiple Image Corruptions
• Authors: Kazuki Adachi, Shin'ya Yamaguchi, Atsutoshi Kumagai
• Abstract summary: Real-world image recognition systems often face corrupted input images, which cause distribution shifts and degrade the performance of models. Test-time adaptation (TTA) is one of the settings that can address this problem. We propose a novel TTA method named Covariance-Aware Feature alignment (CAFe) to address the distribution gap during testing.
• Score: 11.859913430860335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world image recognition systems often face corrupted input images, which cause distribution shifts and degrade the performance of models. These systems often use a single prediction model in a central server and process images sent from various environments, such as cameras distributed in cities or cars. Such single models face images corrupted in heterogeneous ways in test time. Thus, they require to instantly adapt to the multiple corruptions during testing rather than being re-trained at a high cost. Test-time adaptation (TTA), which aims to adapt models without accessing the training dataset, is one of the settings that can address this problem. Existing TTA methods indeed work well on a single corruption. However, the adaptation ability is limited when multiple types of corruption occur, which is more realistic. We hypothesize this is because the distribution shift is more complicated, and the adaptation becomes more difficult in case of multiple corruptions. In fact, we experimentally found that a larger distribution gap remains after TTA. To address the distribution gap during testing, we propose a novel TTA method named Covariance-Aware Feature alignment (CAFe). We empirically show that CAFe outperforms prior TTA methods on image corruptions, including multiple types of corruptions.

Related papers

• Efficient Diffusion-Driven Corruption Editor for Test-Time Adaptation [37.67328706787212]
  Test-time adaptation (TTA) addresses the unforeseen distribution shifts occurring during test time. We propose a novel TTA method that leverages an image editing model based on a latent diffusion model (LDM) and fine-tunes it using our newly introduced corruption modeling scheme. Our model achieves the best performance with a 100 times faster runtime than that of a diffusion-based baseline.
  arXiv  Detail & Related papers  (2024-03-16T12:18:20Z)
• Dynamic Batch Norm Statistics Update for Natural Robustness [5.366500153474747]
  We propose a unified framework consisting of a corruption-detection model and BN statistics update. Our results demonstrate about 8% and 4% accuracy improvement on CIFAR10-C and ImageNet-C.
  arXiv  Detail & Related papers  (2023-10-31T17:20:30Z)
• Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction [3.5139431332194198]
  Test-time adaptation (TTA) continues to update the model after deployment, leveraging the current test data. We identify and highlight several challenges a self-training based method has to deal with. To prevent the model from becoming biased, we leverage a dataset and model-agnostic certainty and diversity weighting.
  arXiv  Detail & Related papers  (2023-06-01T13:16:10Z)
• Ambient Diffusion: Learning Clean Distributions from Corrupted Data [77.34772355241901]
  We present the first diffusion-based framework that can learn an unknown distribution using only highly-corrupted samples. Another benefit of our approach is the ability to train generative models that are less likely to memorize individual training samples.
  arXiv  Detail & Related papers  (2023-05-30T17:43:33Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• On-the-Fly Test-time Adaptation for Medical Image Segmentation [63.476899335138164]
  Adapting the source model to target data distribution at test-time is an efficient solution for the data-shift problem. We propose a new framework called Adaptive UNet where each convolutional block is equipped with an adaptive batch normalization layer. During test-time, the model takes in just the new test image and generates a domain code to adapt the features of source model according to the test data.
  arXiv  Detail & Related papers  (2022-03-10T18:51:29Z)
• SITA: Single Image Test-time Adaptation [48.789568233682296]
  In Test-time Adaptation (TTA), given a model trained on some source data, the goal is to adapt it to make better predictions for test instances from a different distribution. We consider TTA in a more pragmatic setting which we refer to as SITA (Single Image Test-time Adaptation) Here, when making each prediction, the model has access only to the given single test instance, rather than a batch of instances. We propose a novel approach AugBN for the SITA setting that requires only forward-preserving propagation.
  arXiv  Detail & Related papers  (2021-12-04T15:01:35Z)
• Benchmarks for Corruption Invariant Person Re-identification [31.919264399996475]
  We study corruption invariant learning in single- and cross-modality datasets, including Market-1501, CUHK03, MSMT17, RegDB, SYSU-MM01. transformer-based models are more robust towards corrupted images, compared with CNN-based models. Cross-dataset generalization improves with corruption robustness increases.
  arXiv  Detail & Related papers  (2021-11-01T12:14:28Z)
• A Hierarchical Transformation-Discriminating Generative Model for Few Shot Anomaly Detection [93.38607559281601]
  We devise a hierarchical generative model that captures the multi-scale patch distribution of each training image. The anomaly score is obtained by aggregating the patch-based votes of the correct transformation across scales and image regions.
  arXiv  Detail & Related papers  (2021-04-29T17:49:48Z)
• Improving robustness against common corruptions with frequency biased models [112.65717928060195]
  unseen image corruptions can cause a surprisingly large drop in performance. Image corruption types have different characteristics in the frequency spectrum and would benefit from a targeted type of data augmentation. We propose a new regularization scheme that minimizes the total variation (TV) of convolution feature-maps to increase high-frequency robustness.
  arXiv  Detail & Related papers  (2021-03-30T10:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.