Improving Dialogue Breakdown Detection with Semi-Supervised Learning

• URL: http://arxiv.org/abs/2011.00136v1
• Date: Fri, 30 Oct 2020 23:04:56 GMT
• Title: Improving Dialogue Breakdown Detection with Semi-Supervised Learning
• Authors: Nathan Ng and Marzyeh Ghassemi and Narendran Thangarajan and Jiacheng Pan and Qi Guo
• Abstract summary: We investigate the use of semi-supervised learning methods to improve dialogue breakdown detection. We demonstrate the effectiveness of these methods on the Dialogue Breakdown Detection Challenge (DBDC) English shared task.
• Score: 7.7914806980889875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building user trust in dialogue agents requires smooth and consistent dialogue exchanges. However, agents can easily lose conversational context and generate irrelevant utterances. These situations are called dialogue breakdown, where agent utterances prevent users from continuing the conversation. Building systems to detect dialogue breakdown allows agents to recover appropriately or avoid breakdown entirely. In this paper we investigate the use of semi-supervised learning methods to improve dialogue breakdown detection, including continued pre-training on the Reddit dataset and a manifold-based data augmentation method. We demonstrate the effectiveness of these methods on the Dialogue Breakdown Detection Challenge (DBDC) English shared task. Our submissions to the 2020 DBDC5 shared task place first, beating baselines and other submissions by over 12\% accuracy. In ablations on DBDC4 data from 2019, our semi-supervised learning methods improve the performance of a baseline BERT model by 2\% accuracy. These methods are applicable generally to any dialogue task and provide a simple way to improve model performance.

Related papers

• Injecting Salesperson's Dialogue Strategies in Large Language Models with Chain-of-Thought Reasoning [23.919423630938226]
  SalesBot simulates dialogues transitioning from chit-chat to task-oriented scenarios to train sales agents. Initial data lacked smooth transitions and coherent long-turn dialogues, resulting in poor naturalness in sales-customer interactions. We introduce a novel model called SalesAgent, trained on salesperson's interactions, using chain-of-thought (CoT) reasoning.
  arXiv  Detail & Related papers  (2024-04-29T10:12:04Z)
• SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation [55.82577086422923]
  We provide a feasible definition of dialogue segmentation points with the help of document-grounded dialogues. We release a large-scale supervised dataset called SuperDialseg, containing 9,478 dialogues. We also provide a benchmark including 18 models across five categories for the dialogue segmentation task.
  arXiv  Detail & Related papers  (2023-05-15T06:08:01Z)
• CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation [75.60156479374416]
  CGoDial is a new challenging and comprehensive Chinese benchmark for Goal-oriented Dialog evaluation. It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources. To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing.
  arXiv  Detail & Related papers  (2022-11-21T16:21:41Z)
• SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding [68.94808536012371]
  We propose a tree-structured pre-trained conversation model, which learns dialog representations from limited labeled dialogs and large-scale unlabeled dialog corpora. Our method can achieve new state-of-the-art results on the DialoGLUE benchmark consisting of seven datasets and four popular dialog understanding tasks.
  arXiv  Detail & Related papers  (2022-09-14T13:42:50Z)
• OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue [40.62090743056549]
  This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD) Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: dialogue state tracker (DST) and response generator (RG)
  arXiv  Detail & Related papers  (2022-09-10T04:38:27Z)
• In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
  We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST) A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates. This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
  arXiv  Detail & Related papers  (2022-03-16T11:58:24Z)
• Dual Task Framework for Debiasing Persona-grounded Dialogue Dataset [17.403065663306567]
  We introduce a data-centric approach for the task of improving persona-conditioned dialogue agents. Specifically, we augment relevant personas to improve dialogue dataset/agent, by leveraging the primal-dual structure of the two tasks. Experiments on Persona-Chat show that our approach outperforms pre-trained LMs by an 11.7 point gain in terms of accuracy.
  arXiv  Detail & Related papers  (2022-02-11T04:08:46Z)
• Transferable Dialogue Systems and User Simulators [17.106518400787156]
  One of the difficulties in training dialogue systems is the lack of training data. We explore the possibility of creating dialogue data through the interaction between a dialogue system and a user simulator. We develop a modelling framework that can incorporate new dialogue scenarios through self-play between the two agents.
  arXiv  Detail & Related papers  (2021-07-25T22:59:09Z)
• Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Data [61.71319905364992]
  We propose a novel data augmentation method for training open-domain dialogue models by utilizing unpaired data. A data-level distillation process is first proposed to construct augmented dialogues where both post and response are retrieved from the unpaired data. A ranking module is employed to filter out low-quality dialogues. A model-level distillation process is employed to distill a teacher model trained on high-quality paired data to augmented dialogue pairs.
  arXiv  Detail & Related papers  (2020-09-20T13:06:38Z)
• TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue [113.45485470103762]
  In this work, we unify nine human-human and multi-turn task-oriented dialogue datasets for language modeling. To better model dialogue behavior during pre-training, we incorporate user and system tokens into the masked language modeling.
  arXiv  Detail & Related papers  (2020-04-15T04:09:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.