MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention

• URL: http://arxiv.org/abs/2312.05741v1
• Date: Sun, 10 Dec 2023 03:38:41 GMT
• Title: MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention
• Authors: Thinh Pham and Chi Tran and Dat Quoc Nguyen
• Abstract summary: Recent advanced approaches, which are joint models based on graphs, might still face two potential issues. We propose a joint model named MISCA. Our MISCA introduces an intent-slot co-attention mechanism and an underlying layer of label attention mechanism.
• Score: 9.414164374919029
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The research study of detecting multiple intents and filling slots is becoming more popular because of its relevance to complicated real-world situations. Recent advanced approaches, which are joint models based on graphs, might still face two potential issues: (i) the uncertainty introduced by constructing graphs based on preliminary intents and slots, which may transfer intent-slot correlation information to incorrect label node destinations, and (ii) direct incorporation of multiple intent labels for each token w.r.t. token-level intent voting might potentially lead to incorrect slot predictions, thereby hurting the overall performance. To address these two issues, we propose a joint model named MISCA. Our MISCA introduces an intent-slot co-attention mechanism and an underlying layer of label attention mechanism. These mechanisms enable MISCA to effectively capture correlations between intents and slot labels, eliminating the need for graph construction. They also facilitate the transfer of correlation information in both directions: from intents to slots and from slots to intents, through multiple levels of label-specific representations, without relying on token-level intent information. Experimental results show that MISCA outperforms previous models, achieving new state-of-the-art overall accuracy performances on two benchmark datasets MixATIS and MixSNIPS. This highlights the effectiveness of our attention mechanisms.

Related papers

• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• A Dynamic Graph Interactive Framework with Label-Semantic Injection for Spoken Language Understanding [43.48113981442722]
  We propose a framework termed DGIF, which first leverages the semantic information of labels to give the model additional signals and enriched priors. We propose a novel approach to construct the interactive graph based on the injection of label semantics, which can automatically update the graph to better alleviate error propagation.
  arXiv  Detail & Related papers  (2022-11-08T05:57:46Z)
• Bi-directional Joint Neural Networks for Intent Classification and Slot Filling [5.3361357265365035]
  We propose a bi-directional joint model for intent classification and slot filling. Our model achieves state-of-the-art results on intent classification accuracy, slot filling F1, and significantly improves sentence-level semantic frame accuracy.
  arXiv  Detail & Related papers  (2022-02-26T06:35:21Z)
• Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition [65.78703941973183]
  We propose a novel correlation-driven joint-bone fusion graph convolutional network (CD-JBF-GCN) as an encoder and use a pose prediction head as a decoder. Specifically, the CD-JBF-GC can explore the motion transmission between the joint stream and the bone stream. The pose prediction based auto-encoder in the self-supervised training stage allows the network to learn motion representation from unlabeled data.
  arXiv  Detail & Related papers  (2022-02-08T16:03:15Z)
• Towards Joint Intent Detection and Slot Filling via Higher-order Attention [47.78365472691051]
  Intent detection (ID) and Slot filling (SF) are two major tasks in spoken language understanding (SLU) We propose a Bilinear attention block, which exploits both the contextual and channel-wise bilinear attention distributions. We show that our approach yields improvements compared with the state-of-the-art approach.
  arXiv  Detail & Related papers  (2021-09-18T09:50:23Z)
• Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning [51.03781020616402]
  Fine-grained action recognition is attracting increasing attention due to the emerging demand of specific action understanding in real-world applications. We propose a few-shot fine-grained action recognition problem, aiming to recognize novel fine-grained actions with only few samples given for each class. Although progress has been made in coarse-grained actions, existing few-shot recognition methods encounter two issues handling fine-grained actions.
  arXiv  Detail & Related papers  (2021-08-15T02:21:01Z)
• GL-GIN: Fast and Accurate Non-Autoregressive Model for Joint Multiple Intent Detection and Slot Filling [31.833158491112005]
  Multi-intent SLU can handle multiple intents in an utterance, which has attracted increasing attention. In this paper, we explore a non-autoregressive model for joint multiple intent detection and slot filling. Our framework achieves state-of-the-art performance while being 11.5 times faster.
  arXiv  Detail & Related papers  (2021-06-03T15:22:38Z)
• Online Multiple Object Tracking with Cross-Task Synergy [120.70085565030628]
  We propose a novel unified model with synergy between position prediction and embedding association. The two tasks are linked by temporal-aware target attention and distractor attention, as well as identity-aware memory aggregation model.
  arXiv  Detail & Related papers  (2021-04-01T10:19:40Z)
• AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling [69.59096090788125]
  In this paper, we propose an Adaptive Graph-Interactive Framework (AGIF) for joint multiple intent detection and slot filling. We introduce an intent-slot graph interaction layer to model the strong correlation between the slot and intents. Such an interaction layer is applied to each token adaptively, which has the advantage to automatically extract the relevant intents information.
  arXiv  Detail & Related papers  (2020-04-21T15:07:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.