A Diversity-Enhanced and Constraints-Relaxed Augmentation for Low-Resource Classification

• URL: http://arxiv.org/abs/2109.11834v1
• Date: Fri, 24 Sep 2021 09:26:29 GMT
• Title: A Diversity-Enhanced and Constraints-Relaxed Augmentation for Low-Resource Classification
• Authors: Guang Liu, Hailong Huang, Yuzhao Mao, Weiguo Gao, Xuan Li, Jianping Shen
• Abstract summary: In LRC, strong constraints but weak diversity in DA result in the poor ability generalization of classifiers. We propose a Diversity-Enhanced and Constraints-Relaxed Augmentation (DECRA) Our DECRA has two essential components on top of a transformer-based backbone model.
• Score: 8.05097035573437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data augmentation (DA) aims to generate constrained and diversified data to improve classifiers in Low-Resource Classification (LRC). Previous studies mostly use a fine-tuned Language Model (LM) to strengthen the constraints but ignore the fact that the potential of diversity could improve the effectiveness of generated data. In LRC, strong constraints but weak diversity in DA result in the poor generalization ability of classifiers. To address this dilemma, we propose a {D}iversity-{E}nhanced and {C}onstraints-\{R}elaxed {A}ugmentation (DECRA). Our DECRA has two essential components on top of a transformer-based backbone model. 1) A k-beta augmentation, an essential component of DECRA, is proposed to enhance the diversity in generating constrained data. It expands the changing scope and improves the degree of complexity of the generated data. 2) A masked language model loss, instead of fine-tuning, is used as a regularization. It relaxes constraints so that the classifier can be trained with more scattered generated data. The combination of these two components generates data that can reach or approach category boundaries and hence help the classifier generalize better. We evaluate our DECRA on three public benchmark datasets under low-resource settings. Extensive experiments demonstrate that our DECRA outperforms state-of-the-art approaches by 3.8% in the overall score.

Related papers

• Data Augmentation for Image Classification using Generative AI [8.74488498507946]
  Data augmentation is a promising solution to expanding the dataset size. Recent approaches use generative AI models to improve dataset diversity. We propose the Automated Generative Data Augmentation (AGA)
  arXiv  Detail & Related papers  (2024-08-31T21:16:43Z)
• A robust three-way classifier with shadowed granular-balls based on justifiable granularity [53.39844791923145]
  We construct a robust three-way classifier with shadowed GBs for uncertain data. Our model demonstrates in managing uncertain data and effectively mitigates classification risks.
  arXiv  Detail & Related papers  (2024-07-03T08:54:45Z)
• ExaRanker-Open: Synthetic Explanation for IR using Open-Source LLMs [60.81649785463651]
  We introduce ExaRanker-Open, where we adapt and explore the use of open-source language models to generate explanations. Our findings reveal that incorporating explanations consistently enhances neural rankers, with benefits escalating as the LLM size increases.
  arXiv  Detail & Related papers  (2024-02-09T11:23:14Z)
• Detail Reinforcement Diffusion Model: Augmentation Fine-Grained Visual Categorization in Few-Shot Conditions [11.121652649243119]
  Diffusion models have been widely adopted in data augmentation due to their outstanding diversity in data generation. We propose a novel approach termed the detail reinforcement diffusion model(DRDM) It leverages the rich knowledge of large models for fine-grained data augmentation and comprises two key components including discriminative semantic recombination (DSR) and spatial knowledge reference(SKR)
  arXiv  Detail & Related papers  (2023-09-15T01:28:59Z)
• Towards Realistic Low-resource Relation Extraction: A Benchmark with Empirical Baseline Study [51.33182775762785]
  This paper presents an empirical study to build relation extraction systems in low-resource settings. We investigate three schemes to evaluate the performance in low-resource settings: (i) different types of prompt-based methods with few-shot labeled data; (ii) diverse balancing methods to address the long-tailed distribution issue; and (iii) data augmentation technologies and self-training to generate more labeled in-domain data.
  arXiv  Detail & Related papers  (2022-10-19T15:46:37Z)
• CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generation [91.16551253297588]
  COunterfactual Generation via Retrieval and Editing (CORE) is a retrieval-augmented generation framework for creating diverse counterfactual perturbations for training. CORE first performs a dense retrieval over a task-related unlabeled text corpus using a learned bi-encoder. CORE then incorporates these into prompts to a large language model with few-shot learning capabilities, for counterfactual editing.
  arXiv  Detail & Related papers  (2022-10-10T17:45:38Z)
• Augmentation-Aware Self-Supervision for Data-Efficient GAN Training [68.81471633374393]
  Training generative adversarial networks (GANs) with limited data is challenging because the discriminator is prone to overfitting. We propose a novel augmentation-aware self-supervised discriminator that predicts the augmentation parameter of the augmented data. We compare our method with state-of-the-art (SOTA) methods using the class-conditional BigGAN and unconditional StyleGAN2 architectures.
  arXiv  Detail & Related papers  (2022-05-31T10:35:55Z)
• Augmentation-induced Consistency Regularization for Classification [25.388324221293203]
  We propose a consistency regularization framework based on data augmentation, called CR-Aug. CR-Aug forces the output distributions of different sub models generated by data augmentation to be consistent with each other. We implement CR-Aug to image and audio classification tasks and conduct extensive experiments to verify its effectiveness.
  arXiv  Detail & Related papers  (2022-05-25T03:15:36Z)
• Improving Model Compatibility of Generative Adversarial Networks by Boundary Calibration [24.28407308818025]
  Boundary-Calibration GANs (BCGANs) are proposed to improve GAN's model compatibility. BCGANs generate realistic images like original GANs but also achieves superior model compatibility than the original GANs.
  arXiv  Detail & Related papers  (2021-11-03T16:08:09Z)
• Examining and Combating Spurious Features under Distribution Shift [94.31956965507085]
  We define and analyze robust and spurious representations using the information-theoretic concept of minimal sufficient statistics. We prove that even when there is only bias of the input distribution, models can still pick up spurious features from their training data. Inspired by our analysis, we demonstrate that group DRO can fail when groups do not directly account for various spurious correlations.
  arXiv  Detail & Related papers  (2021-06-14T05:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.