Harnessing Hard Mixed Samples with Decoupled Regularizer

• URL: http://arxiv.org/abs/2203.10761v3
• Date: Mon, 23 Oct 2023 16:56:59 GMT
• Title: Harnessing Hard Mixed Samples with Decoupled Regularizer
• Authors: Zicheng Liu, Siyuan Li, Ge Wang, Cheng Tan, Lirong Wu, Stan Z. Li
• Abstract summary: Mixup is an efficient data augmentation approach that improves the generalization of neural networks by smoothing the decision boundary with mixed data. In this paper, we propose an efficient mixup objective function with a decoupled regularizer named Decoupled Mixup (DM) DM can adaptively utilize hard mixed samples to mine discriminative features without losing the original smoothness of mixup.
• Score: 69.98746081734441
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mixup is an efficient data augmentation approach that improves the generalization of neural networks by smoothing the decision boundary with mixed data. Recently, dynamic mixup methods have improved previous static policies effectively (e.g., linear interpolation) by maximizing target-related salient regions in mixed samples, but excessive additional time costs are not acceptable. These additional computational overheads mainly come from optimizing the mixed samples according to the mixed labels. However, we found that the extra optimizing step may be redundant because label-mismatched mixed samples are informative hard mixed samples for deep models to localize discriminative features. In this paper, we thus are not trying to propose a more complicated dynamic mixup policy but rather an efficient mixup objective function with a decoupled regularizer named Decoupled Mixup (DM). The primary effect is that DM can adaptively utilize those hard mixed samples to mine discriminative features without losing the original smoothness of mixup. As a result, DM enables static mixup methods to achieve comparable or even exceed the performance of dynamic methods without any extra computation. This also leads to an interesting objective design problem for mixup training that we need to focus on both smoothing the decision boundaries and identifying discriminative features. Extensive experiments on supervised and semi-supervised learning benchmarks across seven datasets validate the effectiveness of DM as a plug-and-play module. Source code and models are available at https://github.com/Westlake-AI/openmixup

Related papers

• TransformMix: Learning Transformation and Mixing Strategies from Data [20.79680733590554]
  We propose an automated approach, TransformMix, to learn better transformation and mixing augmentation strategies from data. We demonstrate the effectiveness of TransformMix on multiple datasets in transfer learning, classification, object detection, and knowledge distillation settings.
  arXiv  Detail & Related papers  (2024-03-19T04:36:41Z)
• Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
  We introduce a novel convex convex model for semi/library-based unmixing. We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
  arXiv  Detail & Related papers  (2024-01-23T10:07:41Z)
• Adversarial AutoMixup [50.1874436169571]
  We propose AdAutomixup, an adversarial automatic mixup augmentation approach. It generates challenging samples to train a robust classifier for image classification. Our approach outperforms the state of the art in various classification scenarios.
  arXiv  Detail & Related papers  (2023-12-19T08:55:00Z)
• Tailoring Mixup to Data for Calibration [12.050401897136501]
  Mixup is a technique for improving calibration and predictive uncertainty. In this work, we argue that the likelihood of manifold intrusion increases with the distance between data to mix. We propose to dynamically change the underlying distributions of coefficients depending on the similarity between samples to mix.
  arXiv  Detail & Related papers  (2023-11-02T17:48:28Z)
• Tackling Diverse Minorities in Imbalanced Classification [80.78227787608714]
  Imbalanced datasets are commonly observed in various real-world applications, presenting significant challenges in training classifiers. We propose generating synthetic samples iteratively by mixing data samples from both minority and majority classes. We demonstrate the effectiveness of our proposed framework through extensive experiments conducted on seven publicly available benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-28T18:48:34Z)
• C-Mixup: Improving Generalization in Regression [71.10418219781575]
  Mixup algorithm improves generalization by linearly interpolating a pair of examples and their corresponding labels. We propose C-Mixup, which adjusts the sampling probability based on the similarity of the labels. C-Mixup achieves 6.56%, 4.76%, 5.82% improvements in in-distribution generalization, task generalization, and out-of-distribution robustness, respectively.
  arXiv  Detail & Related papers  (2022-10-11T20:39:38Z)
• DoubleMix: Simple Interpolation-Based Data Augmentation for Text Classification [56.817386699291305]
  This paper proposes a simple yet effective data augmentation approach termed DoubleMix. DoubleMix first generates several perturbed samples for each training data. It then uses the perturbed data and original data to carry out a two-step in the hidden space of neural models.
  arXiv  Detail & Related papers  (2022-09-12T15:01:04Z)
• Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity [15.780905917870427]
  We propose a new perspective on batch mixup and formulate the optimal construction of a batch of mixup data. We also propose an efficient modular approximation based iterative submodular computation algorithm for efficient mixup per each minibatch. Our experiments show the proposed method achieves the state of the art generalization, calibration, and weakly supervised localization results.
  arXiv  Detail & Related papers  (2021-02-05T09:12:02Z)
• Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup [19.680580983094323]
  Puzzle Mix is a mixup method for explicitly utilizing the saliency information and the underlying statistics of the natural examples. Our experiments show Puzzle Mix achieves the state of the art generalization and the adversarial robustness results.
  arXiv  Detail & Related papers  (2020-09-15T10:10:23Z)
• MixMOOD: A systematic approach to class distribution mismatch in semi-supervised learning using deep dataset dissimilarity measures [13.823764245165792]
  MixMOOD is a systematic approach to mitigate effect of class distribution mismatch in semi-supervised deep learning (SSDL) with MixMatch. In the first part, we analyze the sensitivity of MixMatch accuracy under 90 different distribution mismatch scenarios. In the second part, we propose an efficient and effective method, called deep dataset dissimilarity measures (DeDiMs) to compare labelled and unlabelled datasets.
  arXiv  Detail & Related papers  (2020-06-14T01:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.