Towards Combating Frequency Simplicity-biased Learning for Domain Generalization

• URL: http://arxiv.org/abs/2410.16146v1
• Date: Mon, 21 Oct 2024 16:17:01 GMT
• Title: Towards Combating Frequency Simplicity-biased Learning for Domain Generalization
• Authors: Xilin He, Jingyu Hu, Qinliang Lin, Cheng Luo, Weicheng Xie, Siyang Song, Muhammad Haris Khan, Linlin Shen,
• Abstract summary: Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets. We propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset.
• Score: 36.777767173275336
• License:
• Abstract: Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization performance. Despite previous data augmentation techniques successfully enhancing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement. In this paper, we aim to prevent such learning behavior of applying frequency shortcuts from a data-driven perspective. Given the theoretical justification of models' biased learning behavior on different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. Intuitively, as frequency shortcuts are hidden in the dominant and highly dependent frequencies of dataset structure, dynamically perturbating the over-reliance frequency components could prevent the application of frequency shortcuts. To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning. Code is available at AdvFrequency (https://github.com/C0notSilly/AdvFrequency).

Related papers

• Tuning Frequency Bias of State Space Models [48.60241978021799]
  State space models (SSMs) leverage linear, time-invariant (LTI) systems to learn sequences with long-range dependencies. We find that SSMs exhibit an implicit bias toward capturing low-frequency components more effectively than high-frequency ones.
  arXiv  Detail & Related papers  (2024-10-02T21:04:22Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• DFM-X: Augmentation by Leveraging Prior Knowledge of Shortcut Learning [3.9858496473361402]
  We propose a data augmentation strategy, named DFM-X, that leverages knowledge about frequency shortcuts. We randomly select X% training images of certain classes for augmentation, and process them by retaining the frequencies included in the DFMs of other classes. Our experimental results demonstrate that DFM-X improves robustness against common corruptions and adversarial attacks.
  arXiv  Detail & Related papers  (2023-08-12T17:39:10Z)
• What do neural networks learn in image classification? A frequency shortcut perspective [3.9858496473361402]
  This study empirically investigates the learning dynamics of frequency shortcuts in neural networks (NNs) We show that NNs tend to find simple solutions for classification, and what they learn first during training depends on the most distinctive frequency characteristics. We propose a metric to measure class-wise frequency characteristics and a method to identify frequency shortcuts.
  arXiv  Detail & Related papers  (2023-07-19T08:34:25Z)
• Towards Building More Robust Models with Frequency Bias [8.510441741759758]
  This paper presents a plug-and-play module that adaptively reconfigures the low- and high-frequency components of intermediate feature representations. Empirical studies show that our proposed module can be easily incorporated into any adversarial training framework.
  arXiv  Detail & Related papers  (2023-07-19T05:46:56Z)
• A Scalable Walsh-Hadamard Regularizer to Overcome the Low-degree Spectral Bias of Neural Networks [79.28094304325116]
  Despite the capacity of neural nets to learn arbitrary functions, models trained through gradient descent often exhibit a bias towards simpler'' functions. We show how this spectral bias towards low-degree frequencies can in fact hurt the neural network's generalization on real-world datasets. We propose a new scalable functional regularization scheme that aids the neural network to learn higher degree frequencies.
  arXiv  Detail & Related papers  (2023-05-16T20:06:01Z)
• Adaptive Frequency Learning in Two-branch Face Forgery Detection [66.91715092251258]
  We propose Adaptively learn Frequency information in the two-branch Detection framework, dubbed AFD. We liberate our network from the fixed frequency transforms, and achieve better performance with our data- and task-dependent transform layers.
  arXiv  Detail & Related papers  (2022-03-27T14:25:52Z)
• Deep Frequency Filtering for Domain Generalization [55.66498461438285]
  Deep Neural Networks (DNNs) have preferences for some frequency components in the learning process. We propose Deep Frequency Filtering (DFF) for learning domain-generalizable features. We show that applying our proposed DFF on a plain baseline outperforms the state-of-the-art methods on different domain generalization tasks.
  arXiv  Detail & Related papers  (2022-03-23T05:19:06Z)
• Understanding robustness and generalization of artificial neural networks through Fourier masks [8.94889125739046]
  Recent literature suggests that robust networks with good generalization properties tend to be biased towards processing low frequencies in images. We develop an algorithm that allows us to learn modulatory masks highlighting the essential input frequencies needed for preserving a trained network's performance.
  arXiv  Detail & Related papers  (2022-03-16T17:32:00Z)
• Robust Learning with Frequency Domain Regularization [1.370633147306388]
  We introduce a new regularization method by constraining the frequency spectra of the filter of the model. We demonstrate the effectiveness of our regularization by (1) defensing to adversarial perturbations; (2) reducing the generalization gap in different architecture; and (3) improving the generalization ability in transfer learning scenario without fine-tune.
  arXiv  Detail & Related papers  (2020-07-07T07:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.