Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling

• URL: http://arxiv.org/abs/2210.10227v1
• Date: Wed, 19 Oct 2022 00:56:10 GMT
• Title: Explainable Slot Type Attentions to Improve Joint Intent Detection and Slot Filling
• Authors: Kalpa Gunaratna, Vijay Srinivasan, Akhila Yerukola, and Hongxia Jin
• Abstract summary: Joint intent detection and slot filling is a key research topic in natural language understanding (NLU) Existing joint intent and slot filling systems analyze and compute features collectively for all slot types. We propose a novel approach that: (i) learns to generate additional slot type specific features in order to improve accuracy and (ii) provides explanations for slot filling decisions for the first time in a joint NLU model.
• Score: 39.22929726787844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Joint intent detection and slot filling is a key research topic in natural language understanding (NLU). Existing joint intent and slot filling systems analyze and compute features collectively for all slot types, and importantly, have no way to explain the slot filling model decisions. In this work, we propose a novel approach that: (i) learns to generate additional slot type specific features in order to improve accuracy and (ii) provides explanations for slot filling decisions for the first time in a joint NLU model. We perform an additional constrained supervision using a set of binary classifiers for the slot type specific feature learning, thus ensuring appropriate attention weights are learned in the process to explain slot filling decisions for utterances. Our model is inherently explainable and does not need any post-hoc processing. We evaluate our approach on two widely used datasets and show accuracy improvements. Moreover, a detailed analysis is also provided for the exclusive slot explainability.

Related papers

• Adaptive Slot Attention: Object Discovery with Dynamic Slot Number [64.45419820717754]
  A major drawback of most object-centric models, including slot attention, is their reliance on predefining the number of slots. Within this framework, we introduce an adaptive slot attention (AdaSlot) mechanism that dynamically determines the optimal number of slots. Our framework, tested extensively on object discovery tasks with various datasets, shows performance matching or exceeding top fixed-slot models.
  arXiv  Detail & Related papers  (2024-06-13T14:55:11Z)
• JPIS: A Joint Model for Profile-based Intent Detection and Slot Filling with Slot-to-Intent Attention [11.12158809959412]
  Profile-based intent detection and slot filling are important tasks aimed at reducing the ambiguity in user utterances. We propose a joint model, namely JPIS, designed to enhance profile-based intent detection and slot filling. Experimental results show that our JPIS substantially outperforms previous profile-based models.
  arXiv  Detail & Related papers  (2023-12-14T08:30:38Z)
• HierarchicalContrast: A Coarse-to-Fine Contrastive Learning Framework for Cross-Domain Zero-Shot Slot Filling [4.1940152307593515]
  Cross-domain zero-shot slot filling plays a vital role in leveraging source domain knowledge to learn a model. Existing state-of-the-art zero-shot slot filling methods have limited generalization ability in target domain. We present a novel Hierarchical Contrastive Learning Framework (HiCL) for zero-shot slot filling.
  arXiv  Detail & Related papers  (2023-10-13T14:23:33Z)
• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• Towards Joint Intent Detection and Slot Filling via Higher-order Attention [47.78365472691051]
  Intent detection (ID) and Slot filling (SF) are two major tasks in spoken language understanding (SLU) We propose a Bilinear attention block, which exploits both the contextual and channel-wise bilinear attention distributions. We show that our approach yields improvements compared with the state-of-the-art approach.
  arXiv  Detail & Related papers  (2021-09-18T09:50:23Z)
• Joint Intent Detection And Slot Filling Based on Continual Learning Model [18.961950574045648]
  A Continual Learning Interrelated Model (CLIM) is proposed to consider semantic information with different characteristics. The experimental results show that CLIM achieves on slot filling and intent detection on ATIS and Snips.
  arXiv  Detail & Related papers  (2021-02-22T11:10:35Z)
• SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling [22.6796529031142]
  Slot filling and intent detection are two main tasks in spoken language understanding (SLU) system. We propose a novel non-autoregressive model named SlotRefine for joint intent detection and slot filling.
  arXiv  Detail & Related papers  (2020-10-06T13:16:53Z)
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
  We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
  arXiv  Detail & Related papers  (2020-09-23T09:45:23Z)
• Attention-based Neural Bag-of-Features Learning for Sequence Data [143.62294358378128]
  2D-Attention (2DA) is a generic attention formulation for sequence data. The proposed attention module is incorporated into the recently proposed Neural Bag of Feature (NBoF) model to enhance its learning capacity. Our empirical analysis shows that the proposed attention formulations can not only improve performances of NBoF models but also make them resilient to noisy data.
  arXiv  Detail & Related papers  (2020-05-25T17:51:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.