Self-Training with Purpose Preserving Augmentation Improves Few-shot Generative Dialogue State Tracking

• URL: http://arxiv.org/abs/2211.09379v1
• Date: Thu, 17 Nov 2022 07:13:58 GMT
• Title: Self-Training with Purpose Preserving Augmentation Improves Few-shot Generative Dialogue State Tracking
• Authors: Jihyun Lee, Chaebin Lee, Yunsu Kim, Gary Geunbae Lee
• Abstract summary: In dialogue state tracking (DST), labeling the dataset involves considerable human labor. We propose a new self-training framework for few-shot generative DST that utilize unlabeled data.
• Score: 14.709084509818474
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In dialogue state tracking (DST), labeling the dataset involves considerable human labor. We propose a new self-training framework for few-shot generative DST that utilize unlabeled data. Our self-training method iteratively improves the model by pseudo labeling and employs Purpose Preserving Augmentation (PPAug) to prevent overfitting. We increaese the few-shot 10% performance by approximately 4% on MultiWOZ 2.1 and enhances the slot-recall 8.34% for unseen values compared to baseline.

Related papers

• Dynamic Noise Preference Optimization for LLM Self-Improvement via Synthetic Data [51.62162460809116]
  We introduce Dynamic Noise Preference Optimization (DNPO) to ensure consistent improvements across iterations. In experiments with Zephyr-7B, DNPO consistently outperforms existing methods, showing an average performance boost of 2.6%. DNPO shows a significant improvement in model-generated data quality, with a 29.4% win-loss rate gap compared to the baseline in GPT-4 evaluations.
  arXiv  Detail & Related papers  (2025-02-08T01:20:09Z)
• EnhancePPG: Improving PPG-based Heart Rate Estimation with Self-Supervision and Augmentation [17.617241860357407]
  We present Enhance, a method that enhances state-of-the-art models by integrating self-supervised learning with data augmentation. Inspired by a U-Net-like autoencoder architecture, we utilize unsupervised PPG signal reconstruction, taking advantage of large amounts of unlabeled data. We improve the best HR estimation by 12.2%, lowering from 4.03 Beats-Per-Minute (BPM) to 3.54 BPM error on PPG-DaLiA.
  arXiv  Detail & Related papers  (2024-12-20T13:25:50Z)
• Silkie: Preference Distillation for Large Visual Language Models [56.10697821410489]
  This paper explores preference distillation for large vision language models (LVLMs) We first build a vision-language feedback dataset utilizing AI annotation. We adopt GPT-4V to assess the generated outputs regarding helpfulness, visual faithfulness, and ethical considerations. The resulting model Silkie, achieves 6.9% and 9.5% relative improvement on the MME benchmark regarding the perception and cognition capabilities.
  arXiv  Detail & Related papers  (2023-12-17T09:44:27Z)
• UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking [54.51316566989655]
  Previous zero-shot dialogue state tracking (DST) methods only apply transfer learning, ignoring unlabelled data in the target domain. We transform zero-shot DST into few-shot DST by utilising such unlabelled data via joint and self-training methods. We demonstrate this method's effectiveness on general language models in zero-shot scenarios, improving average joint goal accuracy by 8% across all domains in MultiWOZ.
  arXiv  Detail & Related papers  (2023-10-16T15:16:16Z)
• CCC-wav2vec 2.0: Clustering aided Cross Contrastive Self-supervised learning of speech representations [1.2031796234206138]
  We present a new pre-training strategy named ccc-wav2vec 2.0, which uses clustering and an augmentation-based cross-contrastive loss as its self-supervised objective. ccc-wav2vec 2.0 achieves up to 15.6% and 12.7% relative WER improvement over the baseline wav2vec 2.0 on the test-clean and test-other sets, respectively, of LibriSpeech, without the use of any language model.
  arXiv  Detail & Related papers  (2022-10-05T22:44:35Z)
• STraTA: Self-Training with Task Augmentation for Better Few-shot Learning [77.04780470527432]
  We propose STraTA, which stands for Self-Training with Task Augmentation. Our experiments demonstrate that STraTA can substantially improve sample efficiency across 12 few-shot benchmarks. Our analyses reveal that task augmentation and self-training are both complementary and independently effective.
  arXiv  Detail & Related papers  (2021-09-13T19:14:01Z)
• Improving Limited Labeled Dialogue State Tracking with Self-Supervision [91.68515201803986]
  Existing dialogue state tracking (DST) models require plenty of labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. Our proposed self-supervised signals can improve joint goal accuracy by 8.95% when only 1% labeled data is used.
  arXiv  Detail & Related papers  (2020-10-26T21:57:42Z)
• Uncertainty-aware Self-training for Text Classification with Few Labels [54.13279574908808]
  We study self-training as one of the earliest semi-supervised learning approaches to reduce the annotation bottleneck. We propose an approach to improve self-training by incorporating uncertainty estimates of the underlying neural network. We show our methods leveraging only 20-30 labeled samples per class for each task for training and for validation can perform within 3% of fully supervised pre-trained language models.
  arXiv  Detail & Related papers  (2020-06-27T08:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.