Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks

• URL: http://arxiv.org/abs/2102.04610v1
• Date: Tue, 9 Feb 2021 02:37:56 GMT
• Title: Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks
• Authors: Pengfei Wei, Bi Zeng and Wenxiong Liao
• Abstract summary: We propose a new joint model with a wheel-graph attention network (Wheel-GAT) which is able to model interrelated connections directly for intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges.
• Score: 6.939768185086755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intent detection and slot filling are two fundamental tasks for building a spoken language understanding (SLU) system. Multiple deep learning-based joint models have demonstrated excellent results on the two tasks. In this paper, we propose a new joint model with a wheel-graph attention network (Wheel-GAT) which is able to model interrelated connections directly for intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges. Intent nodes can provide utterance-level semantic information for slot filling, while slot nodes can also provide local keyword information for intent. Experiments show that our model outperforms multiple baselines on two public datasets. Besides, we also demonstrate that using Bidirectional Encoder Representation from Transformer (BERT) model further boosts the performance in the SLU task.

Related papers

• Co-guiding for Multi-intent Spoken Language Understanding [53.30511968323911]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the mutual guidances between the two tasks. For the first stage, we propose single-task supervised contrastive learning, and for the second stage, we propose co-guiding supervised contrastive learning. Experiment results on multi-intent SLU show that our model outperforms existing models by a large margin.
  arXiv  Detail & Related papers  (2023-11-22T08:06:22Z)
• Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation [4.123763595394021]
  Multiple intent detection and slot filling are fundamental and crucial tasks in spoken language understanding. Joint models that can detect intents and extract slots simultaneously are preferred. We present a method for multiple intent detection and slot filling by addressing these challenges.
  arXiv  Detail & Related papers  (2023-08-28T15:36:33Z)
• Slot Induction via Pre-trained Language Model Probing and Multi-level Contrastive Learning [62.839109775887025]
  Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit unsupervised semantic knowledge extracted from PLM. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-09T05:08:57Z)
• Co-guiding Net: Achieving Mutual Guidances between Multiple Intent Detection and Slot Filling via Heterogeneous Semantics-Label Graphs [39.76268402567324]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the textitmutual guidances between the two tasks. Specifically, we propose two textitheterogeneous graph attention networks working on the proposed two textitheterogeneous semantics-label graphs. Experiment results show that our model outperforms existing models by a large margin, obtaining a relative improvement of 19.3% over the previous best model on MixATIS dataset.
  arXiv  Detail & Related papers  (2022-10-19T08:34:51Z)
• Graph Spring Network and Informative Anchor Selection for Session-based Recommendation [2.6524289609910654]
  Session-based recommendation (SBR) aims at predicting the next item for an ongoing anonymous session. The major challenge of SBR is how to capture richer relations in between items and learn ID-based item embeddings to capture such relations. We propose a new graph neural network, called Graph Spring Network (GSN), for learning ID-based item embedding on an item graph.
  arXiv  Detail & Related papers  (2022-02-19T02:47:44Z)
• Learning Intents behind Interactions with Knowledge Graph for Recommendation [93.08709357435991]
  Knowledge graph (KG) plays an increasingly important role in recommender systems. Existing GNN-based models fail to identify user-item relation at a fine-grained level of intents. We propose a new model, Knowledge Graph-based Intent Network (KGIN)
  arXiv  Detail & Related papers  (2021-02-14T03:21:36Z)
• Node2Seq: Towards Trainable Convolutions in Graph Neural Networks [59.378148590027735]
  We propose a graph network layer, known as Node2Seq, to learn node embeddings with explicitly trainable weights for different neighboring nodes. For a target node, our method sorts its neighboring nodes via attention mechanism and then employs 1D convolutional neural networks (CNNs) to enable explicit weights for information aggregation. In addition, we propose to incorporate non-local information for feature learning in an adaptive manner based on the attention scores.
  arXiv  Detail & Related papers  (2021-01-06T03:05:37Z)
• AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling [69.59096090788125]
  In this paper, we propose an Adaptive Graph-Interactive Framework (AGIF) for joint multiple intent detection and slot filling. We introduce an intent-slot graph interaction layer to model the strong correlation between the slot and intents. Such an interaction layer is applied to each token adaptively, which has the advantage to automatically extract the relevant intents information.
  arXiv  Detail & Related papers  (2020-04-21T15:07:34Z)
• Parallel Intent and Slot Prediction using MLB Fusion [0.0]
  We propose a parallel Intent and Slot Prediction technique where separate Bidirectional Gated Recurrent Units (GRU) are used for each task. We posit the usage of MLB (Multimodal Low-rank Bilinear Attention Network) fusion for improvement in performance of intent and slot learning. Our proposed methods outperform the existing state-of-the-art results for both intent and slot prediction on two benchmark datasets.
  arXiv  Detail & Related papers  (2020-03-20T11:48:16Z)
• Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks [150.5425122989146]
  This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS) AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. Experimental results on three video segmentation datasets show that AGNN sets a new state-of-the-art in each case.
  arXiv  Detail & Related papers  (2020-01-19T10:45:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.