SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolation

• URL: http://arxiv.org/abs/2410.02512v1
• Date: Thu, 3 Oct 2024 14:21:49 GMT
• Title: SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolation
• Authors: Mucong Ding, Bang An, Yuancheng Xu, Anirudh Satheesh, Furong Huang,
• Abstract summary: We present a novel, efficient method for data augmentation, effectively bridging the gap between existing augmentation strategies and emerging datasets and learning tasks. Our findings highlight the potential to adapt existing augmentation pipelines for new data types and tasks, signaling a move towards more adaptable and resilient training frameworks.
• Score: 29.598247232905283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data augmentation, a cornerstone technique in deep learning, is crucial in enhancing model performance, especially with scarce labeled data. While traditional techniques are effective, their reliance on hand-crafted methods limits their applicability across diverse data types and tasks. Although modern learnable augmentation methods offer increased adaptability, they are computationally expensive and challenging to incorporate within prevalent augmentation workflows. In this work, we present a novel, efficient method for data augmentation, effectively bridging the gap between existing augmentation strategies and emerging datasets and learning tasks. We introduce SAFLEX (Self-Adaptive Augmentation via Feature Label EXtrapolation), which learns the sample weights and soft labels of augmented samples provided by any given upstream augmentation pipeline, using a specifically designed efficient bilevel optimization algorithm. Remarkably, SAFLEX effectively reduces the noise and label errors of the upstream augmentation pipeline with a marginal computational cost. As a versatile module, SAFLEX excels across diverse datasets, including natural and medical images and tabular data, showcasing its prowess in few-shot learning and out-of-distribution generalization. SAFLEX seamlessly integrates with common augmentation strategies like RandAug, CutMix, and those from large pre-trained generative models like stable diffusion and is also compatible with frameworks such as CLIP's fine-tuning. Our findings highlight the potential to adapt existing augmentation pipelines for new data types and tasks, signaling a move towards more adaptable and resilient training frameworks.

Related papers

• Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws [59.03420759554073]
  We introduce Adaptive Data Optimization (ADO), an algorithm that optimize data distributions in an online fashion, concurrent with model training. ADO does not require external knowledge, proxy models, or modifications to the model update. ADO uses per-domain scaling laws to estimate the learning potential of each domain during training and adjusts the data mixture accordingly.
  arXiv  Detail & Related papers  (2024-10-15T17:47:44Z)
• Generative Expansion of Small Datasets: An Expansive Graph Approach [13.053285552524052]
  We introduce an Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples. An autoencoder with self-attention layers and optimal transport refines distributional consistency. Results show comparable performance, demonstrating the model's potential to augment training data effectively.
  arXiv  Detail & Related papers  (2024-06-25T02:59:02Z)
• AdaAugment: A Tuning-Free and Adaptive Approach to Enhance Data Augmentation [12.697608744311122]
  AdaAugment is a tuning-free Adaptive Augmentation method. It dynamically adjusts augmentation magnitudes for individual training samples based on real-time feedback from the target network. It consistently outperforms other state-of-the-art DA methods in effectiveness while maintaining remarkable efficiency.
  arXiv  Detail & Related papers  (2024-05-19T06:54:03Z)
• Distribution-Aware Data Expansion with Diffusion Models [55.979857976023695]
  We propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model. DistDiff consistently enhances accuracy across a diverse range of datasets compared to models trained solely on original data.
  arXiv  Detail & Related papers  (2024-03-11T14:07:53Z)
• Incorporating Supervised Domain Generalization into Data Augmentation [4.14360329494344]
  We propose a method, contrastive semantic alignment(CSA) loss, to improve robustness and training efficiency of data augmentation. Experiments on the CIFAR-100 and CUB datasets show that the proposed method improves the robustness and training efficiency of typical data augmentations.
  arXiv  Detail & Related papers  (2023-10-02T09:20:12Z)
• Phased Data Augmentation for Training a Likelihood-Based Generative Model with Limited Data [0.0]
  Generative models excel in creating realistic images, yet their dependency on extensive datasets for training presents significant challenges. Current data-efficient methods largely focus on GAN architectures, leaving a gap in training other types of generative models. "phased data augmentation" is a novel technique that addresses this gap by optimizing training in limited data scenarios without altering the inherent data distribution.
  arXiv  Detail & Related papers  (2023-05-22T03:38:59Z)
• Implicit Counterfactual Data Augmentation for Robust Learning [24.795542869249154]
  This study proposes an Implicit Counterfactual Data Augmentation method to remove spurious correlations and make stable predictions. Experiments have been conducted across various biased learning scenarios covering both image and text datasets.
  arXiv  Detail & Related papers  (2023-04-26T10:36:40Z)
• Automatic Data Augmentation via Invariance-Constrained Learning [94.27081585149836]
  Underlying data structures are often exploited to improve the solution of learning tasks. Data augmentation induces these symmetries during training by applying multiple transformations to the input data. This work tackles these issues by automatically adapting the data augmentation while solving the learning task.
  arXiv  Detail & Related papers  (2022-09-29T18:11:01Z)
• CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signals [92.60744099084157]
  We propose differentiable data augmentation amenable to gradient-based learning. We demonstrate the relevance of our approach on the clinically relevant sleep staging classification task.
  arXiv  Detail & Related papers  (2021-06-25T15:28:48Z)
• Robust Optimization as Data Augmentation for Large-scale Graphs [117.2376815614148]
  We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training. FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks.
  arXiv  Detail & Related papers  (2020-10-19T21:51:47Z)
• CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding [67.61357003974153]
  We propose a novel data augmentation framework dubbed CoDA. CoDA synthesizes diverse and informative augmented examples by integrating multiple transformations organically. A contrastive regularization objective is introduced to capture the global relationship among all the data samples.
  arXiv  Detail & Related papers  (2020-10-16T23:57:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.