Few-shot Intent Classification and Slot Filling with Retrieved Examples

• URL: http://arxiv.org/abs/2104.05763v1
• Date: Mon, 12 Apr 2021 18:50:34 GMT
• Title: Few-shot Intent Classification and Slot Filling with Retrieved Examples
• Authors: Dian Yu and Luheng He and Yuan Zhang and Xinya Du and Panupong Pasupat and Qi Li
• Abstract summary: We propose a span-level retrieval method that learns similar contextualized representations for spans with the same label via a novel batch-softmax objective. Our method outperforms previous systems in various few-shot settings on the CLINC and SNIPS benchmarks.
• Score: 30.45269507626138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Few-shot learning arises in important practical scenarios, such as when a natural language understanding system needs to learn new semantic labels for an emerging, resource-scarce domain. In this paper, we explore retrieval-based methods for intent classification and slot filling tasks in few-shot settings. Retrieval-based methods make predictions based on labeled examples in the retrieval index that are similar to the input, and thus can adapt to new domains simply by changing the index without having to retrain the model. However, it is non-trivial to apply such methods on tasks with a complex label space like slot filling. To this end, we propose a span-level retrieval method that learns similar contextualized representations for spans with the same label via a novel batch-softmax objective. At inference time, we use the labels of the retrieved spans to construct the final structure with the highest aggregated score. Our method outperforms previous systems in various few-shot settings on the CLINC and SNIPS benchmarks.

Related papers

• IntenDD: A Unified Contrastive Learning Approach for Intent Detection and Discovery [12.905097743551774]
  We propose IntenDD, a unified approach leveraging a shared utterance encoding backbone. IntenDD uses an entirely unsupervised contrastive learning strategy for representation learning. We find that our approach consistently outperforms competitive baselines across all three tasks.
  arXiv  Detail & Related papers  (2023-10-25T16:50:24Z)
• CLIP-GCD: Simple Language Guided Generalized Category Discovery [21.778676607030253]
  Generalized Category Discovery (GCD) requires a model to both classify known categories and cluster unknown categories in unlabeled data. Prior methods leveraged self-supervised pre-training combined with supervised fine-tuning on the labeled data, followed by simple clustering methods. We propose to leverage multi-modal (vision and language) models, in two complementary ways.
  arXiv  Detail & Related papers  (2023-05-17T17:55:33Z)
• Out-of-Category Document Identification Using Target-Category Names as Weak Supervision [64.671654559798]
  Out-of-category detection aims to distinguish documents according to their semantic relevance to the inlier (or target) categories. We present an out-of-category detection framework, which effectively measures how confidently each document belongs to one of the target categories.
  arXiv  Detail & Related papers  (2021-11-24T21:01:25Z)
• Revisiting Deep Local Descriptor for Improved Few-Shot Classification [56.74552164206737]
  We show how one can improve the quality of embeddings by leveraging textbfDense textbfClassification and textbfAttentive textbfPooling. We suggest to pool feature maps by applying attentive pooling instead of the widely used global average pooling (GAP) to prepare embeddings for few-shot classification.
  arXiv  Detail & Related papers  (2021-03-30T00:48:28Z)
• Universal Representation Learning from Multiple Domains for Few-shot Classification [41.821234589075445]
  We propose to learn a single set of universal deep representations by distilling knowledge of multiple separately trained networks. We show that the universal representations can be further refined for previously unseen domains by an efficient adaptation step.
  arXiv  Detail & Related papers  (2021-03-25T13:49:12Z)
• Few-shot Learning for Multi-label Intent Detection [59.66787898744991]
  State-of-the-art work estimates label-instance relevance scores and uses a threshold to select multiple associated intent labels. Experiments on two datasets show that the proposed model significantly outperforms strong baselines in both one-shot and five-shot settings.
  arXiv  Detail & Related papers  (2020-10-11T14:42:18Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)
• Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation [132.82884193921535]
  We argue that previous methods underestimate the importance of video feature learning and propose a two-stage approach. We show that this simple baseline approach outperforms prior few-shot video classification methods by over 20 points on existing benchmarks. We present two novel approaches that yield further improvement.
  arXiv  Detail & Related papers  (2020-07-09T13:05:32Z)
• Selecting Relevant Features from a Multi-domain Representation for Few-shot Classification [91.67977602992657]
  We propose a new strategy based on feature selection, which is both simpler and more effective than previous feature adaptation approaches. We show that a simple non-parametric classifier built on top of such features produces high accuracy and generalizes to domains never seen during training.
  arXiv  Detail & Related papers  (2020-03-20T15:44:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.