SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling

• URL: http://arxiv.org/abs/2010.02693v2
• Date: Sat, 31 Oct 2020 12:29:45 GMT
• Title: SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling
• Authors: Di Wu, Liang Ding, Fan Lu and Jian Xie
• Abstract summary: Slot filling and intent detection are two main tasks in spoken language understanding (SLU) system. We propose a novel non-autoregressive model named SlotRefine for joint intent detection and slot filling.
• Score: 22.6796529031142
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Slot filling and intent detection are two main tasks in spoken language understanding (SLU) system. In this paper, we propose a novel non-autoregressive model named SlotRefine for joint intent detection and slot filling. Besides, we design a novel two-pass iteration mechanism to handle the uncoordinated slots problem caused by conditional independence of non-autoregressive model. Experiments demonstrate that our model significantly outperforms previous models in slot filling task, while considerably speeding up the decoding (up to X 10.77). In-depth analyses show that 1) pretraining schemes could further enhance our model; 2) two-pass mechanism indeed remedy the uncoordinated slots.

Related papers

• MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention [9.414164374919029]
  Recent advanced approaches, which are joint models based on graphs, might still face two potential issues. We propose a joint model named MISCA. Our MISCA introduces an intent-slot co-attention mechanism and an underlying layer of label attention mechanism.
  arXiv  Detail & Related papers  (2023-12-10T03:38:41Z)
• Co-guiding for Multi-intent Spoken Language Understanding [53.30511968323911]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the mutual guidances between the two tasks. For the first stage, we propose single-task supervised contrastive learning, and for the second stage, we propose co-guiding supervised contrastive learning. Experiment results on multi-intent SLU show that our model outperforms existing models by a large margin.
  arXiv  Detail & Related papers  (2023-11-22T08:06:22Z)
• Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation [4.123763595394021]
  Multiple intent detection and slot filling are fundamental and crucial tasks in spoken language understanding. Joint models that can detect intents and extract slots simultaneously are preferred. We present a method for multiple intent detection and slot filling by addressing these challenges.
  arXiv  Detail & Related papers  (2023-08-28T15:36:33Z)
• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation [57.11538133231843]
  Keyphrase generation aims to automatically generate short phrases summarizing an input document. The recently emerged ONE2SET paradigm generates keyphrases as a set and has achieved competitive performance. We propose WR-ONE2SET which extends ONE2SET with an adaptive instance-level cost Weighting strategy and a target Re-assignment mechanism.
  arXiv  Detail & Related papers  (2022-11-13T09:56:24Z)
• Are You Stealing My Model? Sample Correlation for Fingerprinting Deep Neural Networks [86.55317144826179]
  Previous methods always leverage the transferable adversarial examples as the model fingerprint. We propose a novel yet simple model stealing detection method based on SAmple Correlation (SAC) SAC successfully defends against various model stealing attacks, even including adversarial training or transfer learning.
  arXiv  Detail & Related papers  (2022-10-21T02:07:50Z)
• Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling [39.22929726787844]
  Joint intent detection and slot filling is a key research topic in natural language understanding (NLU) Existing joint intent and slot filling systems analyze and compute features collectively for all slot types. We propose a novel approach that: (i) learns to generate additional slot type specific features in order to improve accuracy and (ii) provides explanations for slot filling decisions for the first time in a joint NLU model.
  arXiv  Detail & Related papers  (2022-10-19T00:56:10Z)
• FBI: Fingerprinting models with Benign Inputs [17.323638042215013]
  This paper tackles the challenges to propose i) fingerprinting schemes that are resilient to significant modifications of the models, by generalizing to the notion of model families and their variants. We achieve both goals by demonstrating that benign inputs, that are unmodified images, are sufficient material for both tasks. Both approaches are experimentally validated over an unprecedented set of more than 1,000 networks.
  arXiv  Detail & Related papers  (2022-08-05T13:55:36Z)
• GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling [31.833158491112005]
  Multi-intent SLU can handle multiple intents in an utterance, which has attracted increasing attention. In this paper, we explore a non-autoregressive model for joint multiple intent detection and slot filling. Our framework achieves state-of-the-art performance while being 11.5 times faster.
  arXiv  Detail & Related papers  (2021-06-03T15:22:38Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)
• Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling [65.09621991654745]
  Cross-domain slot filling is an essential task in task-oriented dialog systems. We propose a Coarse-to-fine approach (Coach) for cross-domain slot filling. Experimental results show that our model significantly outperforms state-of-the-art approaches in slot filling.
  arXiv  Detail & Related papers  (2020-04-24T13:07:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.