Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening

• URL: http://arxiv.org/abs/2401.14696v1
• Date: Fri, 26 Jan 2024 07:36:57 GMT
• Title: Asymptotic Midpoint Mixup for Margin Balancing and Moderate Broadening
• Authors: Hoyong Kim, Semi Lee, Kangil Kim
• Abstract summary: In the feature space, the collapse between features invokes critical problems in representation learning. We propose a better feature augmentation method, midpoint mixup. We empirically analyze the collapse effects by measuring alignment and uniformity with visualizing representations.
• Score: 4.604003661048267
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In the feature space, the collapse between features invokes critical problems in representation learning by remaining the features undistinguished. Interpolation-based augmentation methods such as mixup have shown their effectiveness in relieving the collapse problem between different classes, called inter-class collapse. However, intra-class collapse raised in coarse-to-fine transfer learning has not been discussed in the augmentation approach. To address them, we propose a better feature augmentation method, asymptotic midpoint mixup. The method generates augmented features by interpolation but gradually moves them toward the midpoint of inter-class feature pairs. As a result, the method induces two effects: 1) balancing the margin for all classes and 2) only moderately broadening the margin until it holds maximal confidence. We empirically analyze the collapse effects by measuring alignment and uniformity with visualizing representations. Then, we validate the intra-class collapse effects in coarse-to-fine transfer learning and the inter-class collapse effects in imbalanced learning on long-tailed datasets. In both tasks, our method shows better performance than other augmentation methods.

Related papers

• Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning [9.712093262192733]
  We propose a simple yet effective method named DisCo to deal with class-incremental learning tasks. DisCo can be easily integrated into existing state-of-the-art class-incremental learning methods. Experimental results show that incorporating our method into various CIL methods achieves substantial performance improvements.
  arXiv  Detail & Related papers  (2024-12-31T03:02:20Z)
• Anti-Collapse Loss for Deep Metric Learning Based on Coding Rate Metric [99.19559537966538]
  DML aims to learn a discriminative high-dimensional embedding space for downstream tasks like classification, clustering, and retrieval. To maintain the structure of embedding space and avoid feature collapse, we propose a novel loss function called Anti-Collapse Loss. Comprehensive experiments on benchmark datasets demonstrate that our proposed method outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-03T13:44:20Z)
• Adaptive Federated Learning Over the Air [108.62635460744109]
  We propose a federated version of adaptive gradient methods, particularly AdaGrad and Adam, within the framework of over-the-air model training. Our analysis shows that the AdaGrad-based training algorithm converges to a stationary point at the rate of $mathcalO( ln(T) / T 1 - frac1alpha ).
  arXiv  Detail & Related papers  (2024-03-11T09:10:37Z)
• Robust Feature Learning and Global Variance-Driven Classifier Alignment for Long-Tail Class Incremental Learning [20.267257778779992]
  This paper introduces a two-stage framework designed to enhance long-tail class incremental learning. We address the challenge posed by the under-representation of tail classes in long-tail class incremental learning. The proposed framework can seamlessly integrate as a module with any class incremental learning method.
  arXiv  Detail & Related papers  (2023-11-02T13:28:53Z)
• Neural Collapse Terminus: A Unified Solution for Class Incremental Learning and Its Variants [166.916517335816]
  In this paper, we offer a unified solution to the misalignment dilemma in the three tasks. We propose neural collapse terminus that is a fixed structure with the maximal equiangular inter-class separation for the whole label space. Our method holds the neural collapse optimality in an incremental fashion regardless of data imbalance or data scarcity.
  arXiv  Detail & Related papers  (2023-08-03T13:09:59Z)
• Improving Adversarial Transferability via Intermediate-level Perturbation Decay [79.07074710460012]
  We develop a novel intermediate-level method that crafts adversarial examples within a single stage of optimization. Experimental results show that it outperforms state-of-the-arts by large margins in attacking various victim models.
  arXiv  Detail & Related papers  (2023-04-26T09:49:55Z)
• Generalized Inter-class Loss for Gait Recognition [11.15855312510806]
  Gait recognition is a unique biometric technique that can be performed at a long distance non-cooperatively. Previous gait works focus more on minimizing the intra-class variance while ignoring the significance in constraining inter-class variance. We propose a generalized inter-class loss which resolves the inter-class variance from both sample-level feature distribution and class-level feature distribution.
  arXiv  Detail & Related papers  (2022-10-13T06:44:53Z)
• Dynamic Feature Regularized Loss for Weakly Supervised Semantic Segmentation [37.43674181562307]
  We propose a new regularized loss which utilizes both shallow and deep features that are dynamically updated. Our approach achieves new state-of-the-art performances, outperforming other approaches by a significant margin with more than 6% mIoU increase.
  arXiv  Detail & Related papers  (2021-08-03T05:11:00Z)
• IntraLoss: Further Margin via Gradient-Enhancing Term for Deep Face Recognition [14.562043494026849]
  Existing classification-based face recognition methods have achieved remarkable progress. Poor feature distribution will wipe out the performance improvement brought about by margin scheme. In this paper, we propose the gradient-enhancing term' that concentrates on the distribution characteristics within the class.
  arXiv  Detail & Related papers  (2021-07-07T16:53:45Z)
• Multi-scale Interactive Network for Salient Object Detection [91.43066633305662]
  We propose the aggregate interaction modules to integrate the features from adjacent levels. To obtain more efficient multi-scale features, the self-interaction modules are embedded in each decoder unit. Experimental results on five benchmark datasets demonstrate that the proposed method without any post-processing performs favorably against 23 state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-07-17T15:41:37Z)
• Towards Certified Robustness of Distance Metric Learning [53.96113074344632]
  We advocate imposing an adversarial margin in the input space so as to improve the generalization and robustness of metric learning algorithms. We show that the enlarged margin is beneficial to the generalization ability by using the theoretical technique of algorithmic robustness.
  arXiv  Detail & Related papers  (2020-06-10T16:51:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.