Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding

• URL: http://arxiv.org/abs/2003.09831v1
• Date: Sun, 22 Mar 2020 07:27:07 GMT
• Title: Prior Knowledge Driven Label Embedding for Slot Filling in Natural Language Understanding
• Authors: Su Zhu, Zijian Zhao, Rao Ma, and Kai Yu
• Abstract summary: This paper proposes a novel label embedding based slot filling framework. Three encoding methods are investigated to incorporate different kinds of prior knowledge about slots. Experiments on single domain and domain adaptation tasks show that label embedding achieves significant performance improvement.
• Score: 28.71402670240912
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional slot filling in natural language understanding (NLU) predicts a one-hot vector for each word. This form of label representation lacks semantic correlation modelling, which leads to severe data sparsity problem, especially when adapting an NLU model to a new domain. To address this issue, a novel label embedding based slot filling framework is proposed in this paper. Here, distributed label embedding is constructed for each slot using prior knowledge. Three encoding methods are investigated to incorporate different kinds of prior knowledge about slots: atomic concepts, slot descriptions, and slot exemplars. The proposed label embeddings tend to share text patterns and reuses data with different slot labels. This makes it useful for adaptive NLU with limited data. Also, since label embedding is independent of NLU model, it is compatible with almost all deep learning based slot filling models. The proposed approaches are evaluated on three datasets. Experiments on single domain and domain adaptation tasks show that label embedding achieves significant performance improvement over traditional one-hot label representation as well as advanced zero-shot approaches.

Related papers

• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels [60.675714333081466]
  Multi-label recognition (MLR) with incomplete labels is very challenging. Recent works strive to explore the image-to-label correspondence in the vision-language model, ie, CLIP, to compensate for insufficient annotations. We advocate remedying the deficiency of label supervision for the MLR with incomplete labels by deriving a structured semantic prior.
  arXiv  Detail & Related papers  (2023-03-23T12:39:20Z)
• Label Name is Mantra: Unifying Point Cloud Segmentation across Heterogeneous Datasets [17.503843467554592]
  We propose a principled approach that supports learning from heterogeneous datasets with different label sets. Our idea is to utilize a pre-trained language model to embed discrete labels to a continuous latent space with the help of their label names. Our model outperforms the state-of-the-art by a large margin.
  arXiv  Detail & Related papers  (2023-03-19T06:14:22Z)
• Enhancing Slot Tagging with Intent Features for Task Oriented Natural Language Understanding using BERT [0.0]
  We examine the effect of leveraging intent label features through 3 techniques in the slot tagging task of joint intent and slot detection models. We evaluate our techniques on benchmark spoken language datasets SNIPS and ATIS, as well as over a large private Bixby dataset.
  arXiv  Detail & Related papers  (2022-05-19T17:41:04Z)
• Joining datasets via data augmentation in the label space for neural networks [6.036150783745836]
  We propose a new technique leveraging artificially created knowledge graph, recurrent neural networks and policy gradient that successfully achieve the dataset joining in the label space. Empirical results on both image and text classification justify the validity of our approach.
  arXiv  Detail & Related papers  (2021-06-17T06:08:11Z)
• Linguistically-Enriched and Context-Aware Zero-shot Slot Filling [6.06746295810681]
  Slot filling is one of the most important challenges in modern task-oriented dialog systems. New domains (i.e., unseen in training) may emerge after deployment. It is imperative that models seamlessly adapt and fill slots from both seen and unseen domains.
  arXiv  Detail & Related papers  (2021-01-16T20:18:16Z)
• A Study on the Autoregressive and non-Autoregressive Multi-label Learning [77.11075863067131]
  We propose a self-attention based variational encoder-model to extract the label-label and label-feature dependencies jointly. Our model can therefore be used to predict all labels in parallel while still including both label-label and label-feature dependencies.
  arXiv  Detail & Related papers  (2020-12-03T05:41:44Z)
• Coach: A Coarse-to-Fine Approach for Cross-domain Slot Filling [65.09621991654745]
  Cross-domain slot filling is an essential task in task-oriented dialog systems. We propose a Coarse-to-fine approach (Coach) for cross-domain slot filling. Experimental results show that our model significantly outperforms state-of-the-art approaches in slot filling.
  arXiv  Detail & Related papers  (2020-04-24T13:07:12Z)
• Density-Aware Graph for Deep Semi-Supervised Visual Recognition [102.9484812869054]
  Semi-supervised learning (SSL) has been extensively studied to improve the generalization ability of deep neural networks for visual recognition. This paper proposes to solve the SSL problem by building a novel density-aware graph, based on which the neighborhood information can be easily leveraged.
  arXiv  Detail & Related papers  (2020-03-30T02:52:40Z)
• Elastic CRFs for Open-ontology Slot Filling [32.17803768259441]
  Slot filling is a crucial component in task-oriented dialog systems that is used to parse (user) utterances into semantic concepts called slots. We propose a new model called elastic conditional random field (eCRF) where each slot is represented by the embedding of its natural language description. New slot values can be detected by eCRF whenever a language description is available for the slot.
  arXiv  Detail & Related papers  (2018-11-04T07:38:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.