Parallel Intent and Slot Prediction using MLB Fusion

• URL: http://arxiv.org/abs/2003.09211v1
• Date: Fri, 20 Mar 2020 11:48:16 GMT
• Title: Parallel Intent and Slot Prediction using MLB Fusion
• Authors: Anmol Bhasin, Bharatram Natarajan, Gaurav Mathur and Himanshu Mangla
• Abstract summary: We propose a parallel Intent and Slot Prediction technique where separate Bidirectional Gated Recurrent Units (GRU) are used for each task. We posit the usage of MLB (Multimodal Low-rank Bilinear Attention Network) fusion for improvement in performance of intent and slot learning. Our proposed methods outperform the existing state-of-the-art results for both intent and slot prediction on two benchmark datasets.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Intent and Slot Identification are two important tasks in Spoken Language Understanding (SLU). For a natural language utterance, there is a high correlation between these two tasks. A lot of work has been done on each of these using Recurrent-Neural-Networks (RNN), Convolution Neural Networks (CNN) and Attention based models. Most of the past work used two separate models for intent and slot prediction. Some of them also used sequence-to-sequence type models where slots are predicted after evaluating the utterance-level intent. In this work, we propose a parallel Intent and Slot Prediction technique where separate Bidirectional Gated Recurrent Units (GRU) are used for each task. We posit the usage of MLB (Multimodal Low-rank Bilinear Attention Network) fusion for improvement in performance of intent and slot learning. To the best of our knowledge, this is the first attempt of using such a technique on text based problems. Also, our proposed methods outperform the existing state-of-the-art results for both intent and slot prediction on two benchmark datasets

Related papers

• Multi-attention Associate Prediction Network for Visual Tracking [3.9628431811908533]
  classification-regression prediction networks have realized impressive success in several modern deep trackers. There is an inherent difference between classification and regression tasks, so they have diverse even opposite demands for feature matching. We propose a multi-attention associate prediction network (MAPNet) to tackle the above problems.
  arXiv  Detail & Related papers  (2024-03-25T03:18:58Z)
• Co-guiding for Multi-intent Spoken Language Understanding [53.30511968323911]
  We propose a novel model termed Co-guiding Net, which implements a two-stage framework achieving the mutual guidances between the two tasks. For the first stage, we propose single-task supervised contrastive learning, and for the second stage, we propose co-guiding supervised contrastive learning. Experiment results on multi-intent SLU show that our model outperforms existing models by a large margin.
  arXiv  Detail & Related papers  (2023-11-22T08:06:22Z)
• Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation [4.123763595394021]
  Multiple intent detection and slot filling are fundamental and crucial tasks in spoken language understanding. Joint models that can detect intents and extract slots simultaneously are preferred. We present a method for multiple intent detection and slot filling by addressing these challenges.
  arXiv  Detail & Related papers  (2023-08-28T15:36:33Z)
• Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks [69.38572074372392]
  We present the first results proving that feature learning occurs during training with a nonlinear model on multiple tasks. Our key insight is that multi-task pretraining induces a pseudo-contrastive loss that favors representations that align points that typically have the same label across tasks.
  arXiv  Detail & Related papers  (2023-07-13T16:39:08Z)
• Refined Edge Usage of Graph Neural Networks for Edge Prediction [51.06557652109059]
  We propose a novel edge prediction paradigm named Edge-aware Message PassIng neuRal nEtworks (EMPIRE) We first introduce an edge splitting technique to specify use of each edge where each edge is solely used as either the topology or the supervision. In order to emphasize the differences between pairs connected by supervision edges and pairs unconnected, we further weight the messages to highlight the relative ones that can reflect the differences.
  arXiv  Detail & Related papers  (2022-12-25T23:19:56Z)
• Effective Cross-Task Transfer Learning for Explainable Natural Language Inference with T5 [50.574918785575655]
  We compare sequential fine-tuning with a model for multi-task learning in the context of boosting performance on two tasks. Our results show that while sequential multi-task learning can be tuned to be good at the first of two target tasks, it performs less well on the second and additionally struggles with overfitting.
  arXiv  Detail & Related papers  (2022-10-31T13:26:08Z)
• A Unified Framework for Multi-intent Spoken Language Understanding with prompting [14.17726194025463]
  We describe a Prompt-based Spoken Language Understanding (PromptSLU) framework, to intuitively unify two sub-tasks into the same form. In detail, ID and SF are completed by concisely filling the utterance into task-specific prompt templates as input, and sharing output formats of key-value pairs sequence. Experiment results show that our framework outperforms several state-of-the-art baselines on two public datasets.
  arXiv  Detail & Related papers  (2022-10-07T05:58:05Z)
• A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis [90.24921443175514]
  We focus on aspect-based sentiment analysis, which involves extracting aspect term, category, and predicting their corresponding polarities. We propose to reformulate the extraction and prediction tasks into the sequence generation task, using a generative language model with unidirectional attention. Our approach outperforms the previous state-of-the-art (based on BERT) on average performance by a large margins in few-shot and full-shot settings.
  arXiv  Detail & Related papers  (2022-04-11T18:31:53Z)
• Retrieval Augmentation to Improve Robustness and Interpretability of Deep Neural Networks [3.0410237490041805]
  In this work, we actively exploit the training data to improve the robustness and interpretability of deep neural networks. Specifically, the proposed approach uses the target of the nearest input example to initialize the memory state of an LSTM model or to guide attention mechanisms. Results show the effectiveness of the proposed models for the two tasks, on the widely used Flickr8 and IMDB datasets.
  arXiv  Detail & Related papers  (2021-02-25T17:38:31Z)
• Joint Intent Detection and Slot Filling with Wheel-Graph Attention Networks [6.939768185086755]
  We propose a new joint model with a wheel-graph attention network (Wheel-GAT) which is able to model interrelated connections directly for intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges.
  arXiv  Detail & Related papers  (2021-02-09T02:37:56Z)
• AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling [69.59096090788125]
  In this paper, we propose an Adaptive Graph-Interactive Framework (AGIF) for joint multiple intent detection and slot filling. We introduce an intent-slot graph interaction layer to model the strong correlation between the slot and intents. Such an interaction layer is applied to each token adaptively, which has the advantage to automatically extract the relevant intents information.
  arXiv  Detail & Related papers  (2020-04-21T15:07:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.