Bi-directional Joint Neural Networks for Intent Classification and Slot Filling

• URL: http://arxiv.org/abs/2202.13079v1
• Date: Sat, 26 Feb 2022 06:35:21 GMT
• Title: Bi-directional Joint Neural Networks for Intent Classification and Slot Filling
• Authors: Soyeon Caren Han, Siqu Long, Huichun Li, Henry Weld, Josiah Poon
• Abstract summary: We propose a bi-directional joint model for intent classification and slot filling. Our model achieves state-of-the-art results on intent classification accuracy, slot filling F1, and significantly improves sentence-level semantic frame accuracy.
• Score: 5.3361357265365035
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intent classification and slot filling are two critical tasks for natural language understanding. Traditionally the two tasks proceeded independently. However, more recently joint models for intent classification and slot filling have achieved state-of-the-art performance, and have proved that there exists a strong relationship between the two tasks. In this paper, we propose a bi-directional joint model for intent classification and slot filling, which includes a multi-stage hierarchical process via BERT and bi-directional joint natural language understanding mechanisms, including intent2slot and slot2intent, to obtain mutual performance enhancement between intent classification and slot filling. The evaluations show that our model achieves state-of-the-art results on intent classification accuracy, slot filling F1, and significantly improves sentence-level semantic frame accuracy when applied to publicly available benchmark datasets, ATIS (88.6%) and SNIPS (92.8%).

Related papers

• MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention [9.414164374919029]
  Recent advanced approaches, which are joint models based on graphs, might still face two potential issues. We propose a joint model named MISCA. Our MISCA introduces an intent-slot co-attention mechanism and an underlying layer of label attention mechanism.
  arXiv  Detail & Related papers  (2023-12-10T03:38:41Z)
• Co-guiding for Multi-intent Spoken Language Understanding [53.30511968323911]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the mutual guidances between the two tasks. For the first stage, we propose single-task supervised contrastive learning, and for the second stage, we propose co-guiding supervised contrastive learning. Experiment results on multi-intent SLU show that our model outperforms existing models by a large margin.
  arXiv  Detail & Related papers  (2023-11-22T08:06:22Z)
• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• An Explicit-Joint and Supervised-Contrastive Learning Framework for Few-Shot Intent Classification and Slot Filling [12.85364483952161]
  Intent classification (IC) and slot filling (SF) are critical building blocks in task-oriented dialogue systems. Few IC/SF models perform well when the number of training samples per class is quite small. We propose a novel explicit-joint and supervised-contrastive learning framework for few-shot intent classification and slot filling.
  arXiv  Detail & Related papers  (2021-10-26T13:28:28Z)
• Concurrent Discrimination and Alignment for Self-Supervised Feature Learning [52.213140525321165]
  Existing self-supervised learning methods learn by means of pretext tasks which are either (1) discriminating that explicitly specify which features should be separated or (2) aligning that precisely indicate which features should be closed together. In this work, we combine the positive aspects of the discriminating and aligning methods, and design a hybrid method that addresses the above issue. Our method explicitly specifies the repulsion and attraction mechanism respectively by discriminative predictive task and concurrently maximizing mutual information between paired views. Our experiments on nine established benchmarks show that the proposed model consistently outperforms the existing state-of-the-art results of self-supervised and transfer
  arXiv  Detail & Related papers  (2021-08-19T09:07:41Z)
• Hierarchical Modeling for Out-of-Scope Domain and Intent Classification [55.23920796595698]
  This paper focuses on out-of-scope intent classification in dialog systems. We propose a hierarchical multi-task learning approach based on a joint model to classify domain and intent simultaneously. Experiments show that the model outperforms existing methods in terms of accuracy, out-of-scope recall and F1.
  arXiv  Detail & Related papers  (2021-04-30T06:38:23Z)
• A survey of joint intent detection and slot-filling models in natural language understanding [0.0]
  This article is a compilation of past work in natural language understanding, especially joint intent classification and slot filling. In this article, we describe trends, approaches, issues, data sets, evaluation metrics in intent classification and slot filling.
  arXiv  Detail & Related papers  (2021-01-20T12:15:11Z)
• PIN: A Novel Parallel Interactive Network for Spoken Language Understanding [68.53121591998483]
  In the existing RNN-based approaches, ID and SF tasks are often jointly modeled to utilize the correlation information between them. The experiments on two benchmark datasets, i.e., SNIPS and ATIS, demonstrate the effectiveness of our approach. More encouragingly, by using the feature embedding of the utterance generated by the pre-trained language model BERT, our method achieves the state-of-the-art among all comparison approaches.
  arXiv  Detail & Related papers  (2020-09-28T15:59:31Z)
• AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling [69.59096090788125]
  In this paper, we propose an Adaptive Graph-Interactive Framework (AGIF) for joint multiple intent detection and slot filling. We introduce an intent-slot graph interaction layer to model the strong correlation between the slot and intents. Such an interaction layer is applied to each token adaptively, which has the advantage to automatically extract the relevant intents information.
  arXiv  Detail & Related papers  (2020-04-21T15:07:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.