Improving Limited Labeled Dialogue State Tracking with Self-Supervision

• URL: http://arxiv.org/abs/2010.13920v1
• Date: Mon, 26 Oct 2020 21:57:42 GMT
• Title: Improving Limited Labeled Dialogue State Tracking with Self-Supervision
• Authors: Chien-Sheng Wu and Steven Hoi and Caiming Xiong
• Abstract summary: Existing dialogue state tracking (DST) models require plenty of labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. Our proposed self-supervised signals can improve joint goal accuracy by 8.95% when only 1% labeled data is used.
• Score: 91.68515201803986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing dialogue state tracking (DST) models require plenty of labeled data. However, collecting high-quality labels is costly, especially when the number of domains increases. In this paper, we address a practical DST problem that is rarely discussed, i.e., learning efficiently with limited labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. We encourage a DST model to have consistent latent distributions given a perturbed input, making it more robust to an unseen scenario. We also add an auxiliary utterance generation task, modeling a potential correlation between conversational behavior and dialogue states. The experimental results show that our proposed self-supervised signals can improve joint goal accuracy by 8.95\% when only 1\% labeled data is used on the MultiWOZ dataset. We can achieve an additional 1.76\% improvement if some unlabeled data is jointly trained as semi-supervised learning. We analyze and visualize how our proposed self-supervised signals help the DST task and hope to stimulate future data-efficient DST research.

Related papers

• Self-Training with Pseudo-Label Scorer for Aspect Sentiment Quad Prediction [54.23208041792073]
  Aspect Sentiment Quad Prediction (ASQP) aims to predict all quads (aspect term, aspect category, opinion term, sentiment polarity) for a given review. A key challenge in the ASQP task is the scarcity of labeled data, which limits the performance of existing methods. We propose a self-training framework with a pseudo-label scorer, wherein a scorer assesses the match between reviews and their pseudo-labels.
  arXiv  Detail & Related papers  (2024-06-26T05:30:21Z)
• UNO-DST: Leveraging Unlabelled Data in Zero-Shot Dialogue State Tracking [54.51316566989655]
  Previous zero-shot dialogue state tracking (DST) methods only apply transfer learning, ignoring unlabelled data in the target domain. We transform zero-shot DST into few-shot DST by utilising such unlabelled data via joint and self-training methods. We demonstrate this method's effectiveness on general language models in zero-shot scenarios, improving average joint goal accuracy by 8% across all domains in MultiWOZ.
  arXiv  Detail & Related papers  (2023-10-16T15:16:16Z)
• Unified Visual Relationship Detection with Vision and Language Models [89.77838890788638]
  This work focuses on training a single visual relationship detector predicting over the union of label spaces from multiple datasets. We propose UniVRD, a novel bottom-up method for Unified Visual Relationship Detection by leveraging vision and language models. Empirical results on both human-object interaction detection and scene-graph generation demonstrate the competitive performance of our model.
  arXiv  Detail & Related papers  (2023-03-16T00:06:28Z)
• Self-Training with Purpose Preserving Augmentation Improves Few-shot Generative Dialogue State Tracking [14.709084509818474]
  In dialogue state tracking (DST), labeling the dataset involves considerable human labor. We propose a new self-training framework for few-shot generative DST that utilize unlabeled data.
  arXiv  Detail & Related papers  (2022-11-17T07:13:58Z)
• CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking [36.18207750352937]
  Few-shot dialogue state tracking (DST) is a realistic problem that trains the DST model with limited labeled data. We propose a few-shot DST framework called CSS, which Combines Self-training and Self-supervised learning methods. Experimental results on the MultiWOZ dataset show that our proposed CSS achieves competitive performance in several few-shot scenarios.
  arXiv  Detail & Related papers  (2022-10-11T04:55:16Z)
• Robust Dialogue State Tracking with Weak Supervision and Sparse Data [2.580163308334609]
  Generalising dialogue state tracking (DST) to new data is challenging due to the strong reliance on abundant and fine-grained supervision during training. Sample sparsity, distributional shift and the occurrence of new concepts and topics frequently lead to severe performance degradation during inference. We propose a training strategy to build extractive DST models without the need for fine-grained manual span labels.
  arXiv  Detail & Related papers  (2022-02-07T16:58:12Z)
• Prompt Learning for Few-Shot Dialogue State Tracking [75.50701890035154]
  This paper focuses on how to learn a dialogue state tracking (DST) model efficiently with limited labeled data. We design a prompt learning framework for few-shot DST, which consists of two main components: value-based prompt and inverse prompt mechanism. Experiments show that our model can generate unseen slots and outperforms existing state-of-the-art few-shot methods.
  arXiv  Detail & Related papers  (2022-01-15T07:37:33Z)
• Unsupervised Domain Adaptive Learning via Synthetic Data for Person Re-identification [101.1886788396803]
  Person re-identification (re-ID) has gained more and more attention due to its widespread applications in video surveillance. Unfortunately, the mainstream deep learning methods still need a large quantity of labeled data to train models. In this paper, we develop a data collector to automatically generate synthetic re-ID samples in a computer game, and construct a data labeler to simultaneously annotate them.
  arXiv  Detail & Related papers  (2021-09-12T15:51:41Z)
• Dual Learning for Dialogue State Tracking [44.679185483585364]
  Dialogue state tracking (DST) is to estimate the dialogue state at each turn. Due to the dependency on complicated dialogue history contexts, DST data annotation is more expensive than single-sentence language understanding. We propose a novel dual-learning framework to make full use of unlabeled data.
  arXiv  Detail & Related papers  (2020-09-22T10:15:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.