Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification

• URL: http://arxiv.org/abs/2009.03695v1
• Date: Tue, 8 Sep 2020 12:39:47 GMT
• Title: Simple is Better! Lightweight Data Augmentation for Low Resource Slot Filling and Intent Classification
• Abstract summary: We show that lightweight augmentation, a set of augmentation methods involving word span and sentence level operations, alleviates data scarcity problems. Experiments on limited data settings show that lightweight augmentation yields significant performance improvement on slot filling. Lightweight augmentation is also beneficial when combined with pre-trained LM-based models, as it improves BERT-based joint intent and slot filling models.
• Score: 3.2996723916635267
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural-based models have achieved outstanding performance on slot filling and intent classification, when fairly large in-domain training data are available. However, as new domains are frequently added, creating sizeable data is expensive. We show that lightweight augmentation, a set of augmentation methods involving word span and sentence level operations, alleviates data scarcity problems. Our experiments on limited data settings show that lightweight augmentation yields significant performance improvement on slot filling on the ATIS and SNIPS datasets, and achieves competitive performance with respect to more complex, state-of-the-art, augmentation approaches. Furthermore, lightweight augmentation is also beneficial when combined with pre-trained LM-based models, as it improves BERT-based joint intent and slot filling models.

Related papers

• HandCraft: Dynamic Sign Generation for Synthetic Data Augmentation [0.0]
  Sign Language Recognition models face significant performance limitations due to insufficient training data availability.<n>We introduce a novel and lightweight sign generation model based on CMLPe.<n>This model, coupled with a synthetic data pretraining approach, consistently improves recognition accuracy.
  arXiv  Detail & Related papers  (2025-08-20T01:38:24Z)
• Raw Data Matters: Enhancing Prompt Tuning by Internal Augmentation on Vision-Language Models [40.43692693102914]
  Augmentation-driven Prompt Tuning (AugPT) is a self-contained distillation-based prompt tuning approach.<n>AugPT employs self-supervised augmentation on unlabeled images in the training set, and introduces a novel gating mechanism based on consensus test.
  arXiv  Detail & Related papers  (2025-08-04T17:59:56Z)
• A Time-Series Data Augmentation Model through Diffusion and Transformer Integration [0.6437284704257459]
  Deep neural networks typically require large volumes of data for training.<n>We propose a simple and effective method that combines the Diffusion and Transformer models.<n>Using the performance improvement of the model after applying augmented data as a benchmark, this approach shows its capability to produce high-quality augmented data.
  arXiv  Detail & Related papers  (2025-05-01T09:40:45Z)
• AugGen: Synthetic Augmentation Can Improve Discriminative Models [14.680260279598045]
  We introduce a novel self-contained synthetic augmentation technique. It strategically samples from a conditional generative model trained exclusively on the target dataset. It achieves 1--12% performance improvements on the IJB-C and IJB-B benchmarks.
  arXiv  Detail & Related papers  (2025-03-14T16:10:21Z)
• Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners [82.72552644267724]
  BoostPFN can outperform standard PFNs with the same size of training samples in large datasets. High performance is maintained for up to 50x of the pre-training size of PFNs.
  arXiv  Detail & Related papers  (2025-03-03T07:31:40Z)
• Little Giants: Synthesizing High-Quality Embedding Data at Scale [71.352883755806]
  We introduce SPEED, a framework that aligns open-source small models to efficiently generate large-scale embedding data. SPEED uses only less than 1/10 of the GPT API calls, outperforming the state-of-the-art embedding model E5_mistral when both are trained solely on their synthetic data.
  arXiv  Detail & Related papers  (2024-10-24T10:47:30Z)
• SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolation [29.598247232905283]
  We present a novel, efficient method for data augmentation, effectively bridging the gap between existing augmentation strategies and emerging datasets and learning tasks. Our findings highlight the potential to adapt existing augmentation pipelines for new data types and tasks, signaling a move towards more adaptable and resilient training frameworks.
  arXiv  Detail & Related papers  (2024-10-03T14:21:49Z)
• AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs [61.13296177652599]
  We show that data mixtures that perform well at smaller scales may not retain their advantage at larger scales. We propose AutoScale, a two-stage, scale-aware data composition framework.
  arXiv  Detail & Related papers  (2024-07-29T17:06:30Z)
• Improving SMOTE via Fusing Conditional VAE for Data-adaptive Noise Filtering [0.5735035463793009]
  We introduce a framework to enhance the SMOTE algorithm using Variational Autoencoders (VAE) Our approach systematically quantifies the density of data points in a low-dimensional latent space using the VAE, simultaneously incorporating information on class labels and classification difficulty. Empirical studies on several imbalanced datasets represent that this simple process innovatively improves the conventional SMOTE algorithm over the deep learning models.
  arXiv  Detail & Related papers  (2024-05-30T07:06:02Z)
• Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks [66.87070857705994]
  In low-resource settings, the amount of seed data samples to use for data augmentation is very small. We propose a novel method that augments training data by incorporating a wealth of examples from other datasets. This approach can ensure that the generated data is not only relevant but also more diverse than what could be achieved using the limited seed data alone.
  arXiv  Detail & Related papers  (2024-02-21T02:45:46Z)
• RandMSAugment: A Mixed-Sample Augmentation for Limited-Data Scenarios [3.018656336329545]
  We introduce a novel augmentation technique called RandMSAugment that integrates complementary strengths of existing methods. RandMSAugment significantly outperforms the competition on CIFAR-100, STL-10, and Tiny-Imagenet.
  arXiv  Detail & Related papers  (2023-11-25T18:07:20Z)
• DualAug: Exploiting Additional Heavy Augmentation with OOD Data Rejection [77.6648187359111]
  We propose a novel data augmentation method, named textbfDualAug, to keep the augmentation in distribution as much as possible at a reasonable time and computational cost. Experiments on supervised image classification benchmarks show that DualAug improve various automated data augmentation method.
  arXiv  Detail & Related papers  (2023-10-12T08:55:10Z)
• CATfOOD: Counterfactual Augmented Training for Improving Out-of-Domain Performance and Calibration [59.48235003469116]
  We show that data augmentation consistently enhances OOD performance. We also show that CF augmented models which are easier to calibrate also exhibit much lower entropy when assigning importance.
  arXiv  Detail & Related papers  (2023-09-14T16:16:40Z)
• To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis [50.31589712761807]
  Large language models (LLMs) are notoriously token-hungry during pre-training, and high-quality text data on the web is approaching its scaling limit for LLMs. We investigate the consequences of repeating pre-training data, revealing that the model is susceptible to overfitting. Second, we examine the key factors contributing to multi-epoch degradation, finding that significant factors include dataset size, model parameters, and training objectives.
  arXiv  Detail & Related papers  (2023-05-22T17:02:15Z)
• ScoreMix: A Scalable Augmentation Strategy for Training GANs with Limited Data [93.06336507035486]
  Generative Adversarial Networks (GANs) typically suffer from overfitting when limited training data is available. We present ScoreMix, a novel and scalable data augmentation approach for various image synthesis tasks.
  arXiv  Detail & Related papers  (2022-10-27T02:55:15Z)
• On Automatic Data Augmentation for 3D Point Cloud Classification [19.338266486983176]
  We propose to automatically learn a data augmentation strategy using bilevel optimization. An augmentor is designed in a similar fashion to a conditional generator and is optimized by minimizing a base model's loss on a validation set. We evaluate our approach on standard point cloud classification tasks and a more challenging setting with pose misalignment between training and validation/test sets.
  arXiv  Detail & Related papers  (2021-12-11T17:14:16Z)
• Data balancing for boosting performance of low-frequency classes in Spoken Language Understanding [9.689893038619585]
  This paper presents the first systematic study on handling data imbalance for Spoken Language Understanding (SLU) applications. We discuss the application of existing data balancing techniques for SLU and propose a multi-task SLU model for intent classification and slot filling. Our results on a real-world dataset indicate that i) our proposed model can boost performance on low frequency intents significantly while avoiding a potential performance decrease on the head intents, and ii) synthetic data are beneficial for bootstrapping new intents when realistic data are not available.
  arXiv  Detail & Related papers  (2020-08-06T12:23:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.