Epsilon Consistent Mixup: An Adaptive Consistency-Interpolation Tradeoff

• URL: http://arxiv.org/abs/2104.09452v1
• Date: Mon, 19 Apr 2021 17:10:31 GMT
• Title: Epsilon Consistent Mixup: An Adaptive Consistency-Interpolation Tradeoff
• Authors: Vincent Pisztora, Yanglan Ou, Xiaolei Huang, Francesca Chiaromonte, Jia Li
• Abstract summary: $epsilon$mu is a data-based structural regularization technique that combines Mixup's linear with consistency regularization in the Mixup direction. It is shown to improve semi-supervised classification accuracy on the SVHN and CIFAR10 benchmark datasets. In particular, $epsilon$mu is found to produce more accurate synthetic labels and more confident predictions than Mixup.
• Score: 19.03167022268852
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we propose $\epsilon$-Consistent Mixup ($\epsilon$mu). $\epsilon$mu is a data-based structural regularization technique that combines Mixup's linear interpolation with consistency regularization in the Mixup direction, by compelling a simple adaptive tradeoff between the two. This learnable combination of consistency and interpolation induces a more flexible structure on the evolution of the response across the feature space and is shown to improve semi-supervised classification accuracy on the SVHN and CIFAR10 benchmark datasets, yielding the largest gains in the most challenging low label-availability scenarios. Empirical studies comparing $\epsilon$mu and Mixup are presented and provide insight into the mechanisms behind $\epsilon$mu's effectiveness. In particular, $\epsilon$mu is found to produce more accurate synthetic labels and more confident predictions than Mixup.

Related papers

• Fast Semisupervised Unmixing Using Nonconvex Optimization [80.11512905623417]
  We introduce a novel convex convex model for semi/library-based unmixing. We demonstrate the efficacy of Alternating Methods of sparse unsupervised unmixing.
  arXiv  Detail & Related papers  (2024-01-23T10:07:41Z)
• Tailoring Mixup to Data for Calibration [12.050401897136501]
  Mixup is a technique for improving calibration and predictive uncertainty. In this work, we argue that the likelihood of manifold intrusion increases with the distance between data to mix. We propose to dynamically change the underlying distributions of coefficients depending on the similarity between samples to mix.
  arXiv  Detail & Related papers  (2023-11-02T17:48:28Z)
• SIGMA: Scale-Invariant Global Sparse Shape Matching [50.385414715675076]
  We propose a novel mixed-integer programming (MIP) formulation for generating precise sparse correspondences for non-rigid shapes. We show state-of-the-art results for sparse non-rigid matching on several challenging 3D datasets.
  arXiv  Detail & Related papers  (2023-08-16T14:25:30Z)
• Semantic Equivariant Mixup [54.734054770032934]
  Mixup is a well-established data augmentation technique, which can extend the training distribution and regularize the neural networks. Previous mixup variants tend to over-focus on the label-related information. We propose a semantic equivariant mixup (sem) to preserve richer semantic information in the input.
  arXiv  Detail & Related papers  (2023-08-12T03:05:53Z)
• Provable Benefit of Mixup for Finding Optimal Decision Boundaries [18.668531108219415]
  We investigate how pair-wise data augmentation techniques like Mixup affect the sample complexity of finding optimal decision boundaries. We show that Mixup mitigates this problem by significantly reducing the sample complexity.
  arXiv  Detail & Related papers  (2023-06-01T00:59:19Z)
• C-Mixup: Improving Generalization in Regression [71.10418219781575]
  Mixup algorithm improves generalization by linearly interpolating a pair of examples and their corresponding labels. We propose C-Mixup, which adjusts the sampling probability based on the similarity of the labels. C-Mixup achieves 6.56%, 4.76%, 5.82% improvements in in-distribution generalization, task generalization, and out-of-distribution robustness, respectively.
  arXiv  Detail & Related papers  (2022-10-11T20:39:38Z)
• Multi-Sample $\zeta$-mixup: Richer, More Realistic Synthetic Samples from a $p$-Series Interpolant [16.65329510916639]
  We propose $zeta$-mixup, a generalization of mixup with provably and demonstrably desirable properties. We show that our implementation of $zeta$-mixup is faster than mixup, and extensive evaluation on controlled synthetic and 24 real-world natural and medical image classification datasets shows that $zeta$-mixup outperforms mixup and traditional data augmentation techniques.
  arXiv  Detail & Related papers  (2022-04-07T09:41:09Z)
• Harnessing Hard Mixed Samples with Decoupled Regularizer [69.98746081734441]
  Mixup is an efficient data augmentation approach that improves the generalization of neural networks by smoothing the decision boundary with mixed data. In this paper, we propose an efficient mixup objective function with a decoupled regularizer named Decoupled Mixup (DM) DM can adaptively utilize hard mixed samples to mine discriminative features without losing the original smoothness of mixup.
  arXiv  Detail & Related papers  (2022-03-21T07:12:18Z)
• k-Mixup Regularization for Deep Learning via Optimal Transport [32.951696405505686]
  Mixup is a popular regularization technique for training deep neural networks. We extend mixup in a simple, broadly applicable way to emph$k$-mixup, which perturbs $k$-batches of training points in the direction of other $k$-batches. We show that training with $k$-mixup further improves generalization and robustness across several network architectures.
  arXiv  Detail & Related papers  (2021-06-05T17:08:08Z)
• On Mixup Regularization [16.748910388577308]
  Mixup is a data augmentation technique that creates new examples as convex combinations of training points and labels. We show how the random perturbation of the new interpretation of Mixup induces multiple known regularization schemes.
  arXiv  Detail & Related papers  (2020-06-10T20:11:46Z)
• Consistency Regularization for Certified Robustness of Smoothed Classifiers [89.72878906950208]
  A recent technique of randomized smoothing has shown that the worst-case $ell$-robustness can be transformed into the average-case robustness. We found that the trade-off between accuracy and certified robustness of smoothed classifiers can be greatly controlled by simply regularizing the prediction consistency over noise.
  arXiv  Detail & Related papers  (2020-06-07T06:57:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.