Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System

• URL: http://arxiv.org/abs/2305.02468v3
• Date: Wed, 31 May 2023 06:18:28 GMT
• Title: Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System
• Authors: Namo Bang, Jeehyun Lee, Myoung-Wan Koo
• Abstract summary: We propose an End-to-end TOD system with Task-d Adapters which learn independently per task, adding only small number of parameters after fixed layers of pre-trained network. Our method is a model-agnostic approach and does not require prompt-tuning as only input data without a prompt.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Task-Oriented Dialogue (TOD) systems are designed to carry out specific tasks by tracking dialogue states and generating appropriate responses to help users achieve defined goals. Recently, end-to-end dialogue models pre-trained based on large datasets have shown promising performance in the conversational system. However, they share the same parameters to train tasks of the dialogue system (NLU, DST, NLG), so debugging each task is challenging. Also, they require a lot of effort to fine-tune large parameters to create a task-oriented chatbot, making it difficult for non-experts to handle. Therefore, we intend to train relatively lightweight and fast models compared to PLM. In this paper, we propose an End-to-end TOD system with Task-Optimized Adapters which learn independently per task, adding only small number of parameters after fixed layers of pre-trained network. We also enhance the performance of the DST and NLG modules through reinforcement learning, overcoming the learning curve that has lacked at the adapter learning and enabling the natural and consistent response generation that is appropriate for the goal. Our method is a model-agnostic approach and does not require prompt-tuning as only input data without a prompt. As results of the experiment, our method shows competitive performance on the MultiWOZ benchmark compared to the existing end-to-end models. In particular, we attain state-of-the-art performance on the DST task of 2.2 dataset.

Related papers

• Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems [25.14460456391397]
  Large language model (LLM) based TOD systems can excel even with limited data due to their ability to learn tasks through in-context exemplars. We propose SyncTOD that synergizes LLMs with task-specific hints to improve alignment in low-data settings. With ChatGPT, SyncTOD achieves superior performance compared to LLM-based baselines and SoTA models in low-data settings.
  arXiv  Detail & Related papers  (2024-05-24T14:13:54Z)
• Many Hands Make Light Work: Task-Oriented Dialogue System with Module-Based Mixture-of-Experts [9.129081545049992]
  Task-oriented dialogue systems have greatly benefited from pre-trained language models (PLMs) We propose Soft Mixture-of-Expert Task-Oriented Dialogue system (SMETOD) SMETOD leverages an ensemble of Mixture-of-Experts (MoEs) to excel at subproblems and generate specialized outputs for task-oriented dialogues. We extensively evaluate our model on three benchmark functionalities: intent prediction, dialogue state tracking, and dialogue response generation.
  arXiv  Detail & Related papers  (2024-05-16T01:02:09Z)
• Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System [89.40590076430297]
  This work enables the TOD systems with more flexibility through a simple cache. We train end-to-end TOD models that can refer to and ground on both dialogue history and retrieved information during TOD generation. Experiments demonstrate the superior performance of our framework, with a notable improvement in non-empty joint goal accuracy by 6.7% compared to strong baselines.
  arXiv  Detail & Related papers  (2023-08-16T06:52:10Z)
• Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking [57.92608483099916]
  Large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks. For more complex tasks such as dialogue state tracking (DST), designing prompts that reliably convey the desired intent is nontrivial. We introduce a saliency model to limit dialogue text length, allowing us to include more exemplars per query.
  arXiv  Detail & Related papers  (2023-02-12T15:05:10Z)
• OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue [40.62090743056549]
  This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD) Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: dialogue state tracker (DST) and response generator (RG)
  arXiv  Detail & Related papers  (2022-09-10T04:38:27Z)
• Instance-wise Prompt Tuning for Pretrained Language Models [72.74916121511662]
  Instance-wise Prompt Tuning (IPT) is the first prompt learning paradigm that injects knowledge from the input data instances to the prompts. IPT significantly outperforms task-based prompt learning methods, and achieves comparable performance to conventional finetuning with only 0.5% - 1.5% of tuned parameters.
  arXiv  Detail & Related papers  (2022-06-04T10:08:50Z)
• Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System [26.837972034630003]
  PPTOD is a unified plug-and-play model for task-oriented dialogue. We extensively test our model on three benchmark TOD tasks, including end-to-end dialogue modelling, dialogue state tracking, and intent classification.
  arXiv  Detail & Related papers  (2021-09-29T22:02:18Z)
• CINS: Comprehensive Instruction for Few-shot Learning in Task-oriented Dialog Systems [56.302581679816775]
  This paper proposes Comprehensive Instruction (CINS) that exploits PLMs with task-specific instructions. We design a schema (definition, constraint, prompt) of instructions and their customized realizations for three important downstream tasks in ToD. Experiments are conducted on these ToD tasks in realistic few-shot learning scenarios with small validation data.
  arXiv  Detail & Related papers  (2021-09-10T03:23:06Z)
• Language Models as Few-Shot Learner for Task-Oriented Dialogue Systems [74.8759568242933]
  Task-oriented dialogue systems use four connected modules, namely, Natural Language Understanding (NLU), a Dialogue State Tracking (DST), Dialogue Policy (DP) and Natural Language Generation (NLG) A research challenge is to learn each module with the least amount of samples given the high cost related to the data collection. We evaluate the priming few-shot ability of language models in the NLU, DP and NLG tasks.
  arXiv  Detail & Related papers  (2020-08-14T08:23:21Z)
• SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching [81.45928589522032]
  We parameterize modular task-oriented dialog systems using a Transformer-based auto-regressive language model. We pre-train, on heterogeneous dialog corpora, a task-grounded response generation model. Experiments show that SOLOIST creates new state-of-the-art on well-studied task-oriented dialog benchmarks.
  arXiv  Detail & Related papers  (2020-05-11T17:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.