Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models

• URL: http://arxiv.org/abs/2108.08451v1
• Date: Thu, 19 Aug 2021 02:52:40 GMT
• Title: Augmenting Slot Values and Contexts for Spoken Language Understanding with Pretrained Models
• Authors: Haitao Lin, Lu Xiang, Yu Zhou, Jiajun Zhang, Chengqing Zong
• Abstract summary: Spoken Language Understanding (SLU) is one essential step in building a dialogue system. Due to the expensive cost of obtaining the labeled data, SLU suffers from the data scarcity problem. We propose two strategies for finetuning process: value-based and context-based augmentation.
• Score: 45.477765875738115
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spoken Language Understanding (SLU) is one essential step in building a dialogue system. Due to the expensive cost of obtaining the labeled data, SLU suffers from the data scarcity problem. Therefore, in this paper, we focus on data augmentation for slot filling task in SLU. To achieve that, we aim at generating more diverse data based on existing data. Specifically, we try to exploit the latent language knowledge from pretrained language models by finetuning them. We propose two strategies for finetuning process: value-based and context-based augmentation. Experimental results on two public SLU datasets have shown that compared with existing data augmentation methods, our proposed method can generate more diverse sentences and significantly improve the performance on SLU.

Related papers

• Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning [50.1035273069458]
  Spoken language understanding (SLU) is a core task in task-oriented dialogue systems. We propose a multi-level MMCL framework to apply contrastive learning at three levels, including utterance level, slot level, and word level. Our framework achieves new state-of-the-art results on two public multi-intent SLU datasets.
  arXiv  Detail & Related papers  (2024-05-31T14:34:23Z)
• I$^2$KD-SLU: An Intra-Inter Knowledge Distillation Framework for Zero-Shot Cross-Lingual Spoken Language Understanding [1.223779595809275]
  We propose an Intra-Inter Knowledge Distillation framework for zero-shot cross-lingual Spoken Language Understanding (I$2$KD-SLU) Specifically, we not only apply intra-knowledge distillation between intent predictions or slot predictions of the same utterance in different languages, but also apply inter-knowledge distillation between intent predictions and slot predictions of the same utterance. Our experimental results demonstrate that our proposed framework significantly improves the performance compared with the strong baselines.
  arXiv  Detail & Related papers  (2023-10-04T05:45:23Z)
• LaDA: Latent Dialogue Action For Zero-shot Cross-lingual Neural Network Language Modeling [20.002861239367704]
  Cross-lingual adaptation has proven effective in spoken language understanding systems with limited resources. Existing methods are frequently unsatisfactory for intent detection and slot filling. Latent Dialogue Action layer is proposed to optimize decoding strategy.
  arXiv  Detail & Related papers  (2023-08-05T15:51:45Z)
• Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target [58.59044226658916]
  Spoken Language Understanding (SLU) is a task that aims to extract semantic information from spoken utterances. We propose to use discrete units as intermediate guidance to improve textless SLU performance.
  arXiv  Detail & Related papers  (2023-05-29T14:00:24Z)
• STOP: A dataset for Spoken Task Oriented Semantic Parsing [66.14615249745448]
  End-to-end spoken language understanding (SLU) predicts intent directly from audio using a single model. We release the Spoken Task-Oriented semantic Parsing (STOP) dataset, the largest and most complex SLU dataset to be publicly available. In addition to the human-recorded audio, we are releasing a TTS-generated version to benchmark the performance for low-resource domain adaptation of end-to-end SLU systems.
  arXiv  Detail & Related papers  (2022-06-29T00:36:34Z)
• Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding [69.40915115518523]
  Lack of training data presents a grand challenge to scaling out spoken language understanding (SLU) to low-resource languages. Various data augmentation approaches have been proposed to synthesize training data in low-resource target languages. In this paper we focus on mitigating noise in augmented data.
  arXiv  Detail & Related papers  (2021-09-03T15:44:15Z)
• Data Augmentation for Spoken Language Understanding via Pretrained Language Models [113.56329266325902]
  Training of spoken language understanding (SLU) models often faces the problem of data scarcity. We put forward a data augmentation method using pretrained language models to boost the variability and accuracy of generated utterances.
  arXiv  Detail & Related papers  (2020-04-29T04:07:12Z)
• Transfer Learning for Context-Aware Spoken Language Understanding [3.763434958496263]
  Spoken language understanding (SLU) is a key component of task-oriented dialogue systems. Previous work has shown that incorporating context information significantly improves SLU performance for multi-turn dialogues. We propose a Context adaptive Language Transformer (CELT) model facilitating exploiting various context information for SLU.
  arXiv  Detail & Related papers  (2020-03-03T02:56:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.