SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU

• URL: http://arxiv.org/abs/2311.09502v2
• Date: Mon, 8 Apr 2024 21:31:57 GMT
• Title: SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLU
• Authors: Evgeniia Razumovskaia, Goran Glavaš, Anna Korhonen, Ivan Vulić,
• Abstract summary: SQATIN is a new framework for dialog NLU based on (i) instruction tuning and (ii) question-answering-based formulation of ID and VE tasks. SQATIN sets the new state of the art in dialogue NLU, substantially surpassing the performance of current models.
• Score: 21.805799634495486
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Task-oriented dialogue (ToD) systems help users execute well-defined tasks across a variety of domains (e.g., $\textit{flight booking}$ or $\textit{food ordering}$), with their Natural Language Understanding (NLU) components being dedicated to the analysis of user utterances, predicting users' intents ($\textit{Intent Detection}$, ID) and extracting values for informational slots ($\textit{Value Extraction}$, VE). In most domains, labelled NLU data is scarce, making sample-efficient learning -- enabled with effective transfer paradigms -- paramount. In this work, we introduce SQATIN, a new framework for dialog NLU based on (i) instruction tuning and (ii) question-answering-based formulation of ID and VE tasks. According to the evaluation on established NLU benchmarks, SQATIN sets the new state of the art in dialogue NLU, substantially surpassing the performance of current models based on standard fine-tuning objectives in both in-domain training and cross-domain transfer. SQATIN yields particularly large performance gains in cross-domain transfer, owing to the fact that our QA-based instruction tuning leverages similarities between natural language descriptions of classes (i.e., slots and intents) across domains.

Related papers

• The Task-oriented Queries Benchmark (ToQB) [0.0]
  A standard benchmark for task-oriented queries is not yet available. Existing benchmarks in the relevant NLP fields have primarily focused on task-oriented dialogues.
  arXiv  Detail & Related papers  (2024-06-05T05:05:41Z)
• HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Models [13.963676467274109]
  We extend the capabilities of HELPER by expanding its memory with a wider array of examples and prompts. This simple expansion of HELPER into a shared memory enables the agent to work across domains executing plans from dialogue, natural language instruction, active question asking, and common room reorganization. We evaluate the agent on four diverse interactive visual-language embodied agent: AChRED, TEA, DialFRED, and the Tidy Task.
  arXiv  Detail & Related papers  (2024-04-29T19:12:42Z)
• MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answering [64.6741991162092]
  We present MinPrompt, a minimal data augmentation framework for open-domain question answering. We transform the raw text into a graph structure to build connections between different factual sentences. We then apply graph algorithms to identify the minimal set of sentences needed to cover the most information in the raw text. We generate QA pairs based on the identified sentence subset and train the model on the selected sentences to obtain the final model.
  arXiv  Detail & Related papers  (2023-10-08T04:44:36Z)
• Towards Unified Token Learning for Vision-Language Tracking [65.96561538356315]
  We present a vision-language (VL) tracking pipeline, termed textbfMMTrack, which casts VL tracking as a token generation task. Our proposed framework serializes language description and bounding box into a sequence of discrete tokens. In this new design paradigm, all token queries are required to perceive the desired target and directly predict spatial coordinates of the target.
  arXiv  Detail & Related papers  (2023-08-27T13:17:34Z)
• SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding [103.34092301324425]
  Large language models (LLMs) have shown impressive ability for open-domain NLP tasks. We present SeqGPT, a bilingual (i.e., English and Chinese) open-source autoregressive model specially enhanced for open-domain natural language understanding.
  arXiv  Detail & Related papers  (2023-08-21T07:31:19Z)
• SLUE Phase-2: A Benchmark Suite of Diverse Spoken Language Understanding Tasks [88.4408774253634]
  Spoken language understanding (SLU) tasks have been studied for many decades in the speech research community. There are not nearly as many SLU task benchmarks, and many of the existing ones use data that is not freely available to all researchers. Recent work has begun to introduce such benchmark for several tasks.
  arXiv  Detail & Related papers  (2022-12-20T18:39:59Z)
• NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialogue [53.54788957697192]
  NLU++ is a novel dataset for natural language understanding (NLU) in task-oriented dialogue (ToD) systems. NLU++ is divided into two domains (BANKING and HOTELS) and brings several crucial improvements over current commonly used NLU datasets.
  arXiv  Detail & Related papers  (2022-04-27T16:00:23Z)
• On the Use of External Data for Spoken Named Entity Recognition [40.93448412171246]
  Recent advances in self-supervised speech representations have made it feasible to consider learning models with limited labeled data. We draw on a variety of approaches, including self-training, knowledge distillation, and transfer learning, and consider their applicability to both end-to-end models and pipeline approaches.
  arXiv  Detail & Related papers  (2021-12-14T18:49:26Z)
• Zero-Shot Dialogue State Tracking via Cross-Task Transfer [69.70718906395182]
  We propose to transfer the textitcross-task knowledge from general question answering (QA) corpora for the zero-shot dialogue state tracking task. Specifically, we propose TransferQA, a transferable generative QA model that seamlessly combines extractive QA and multi-choice QA. In addition, we introduce two effective ways to construct unanswerable questions, namely, negative question sampling and context truncation.
  arXiv  Detail & Related papers  (2021-09-10T03:57:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.