Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling

• URL: http://arxiv.org/abs/2106.07343v1
• Date: Tue, 25 May 2021 15:07:11 GMT
• Title: Learning to Bridge Metric Spaces: Few-shot Joint Learning of Intent Detection and Slot Filling
• Authors: Yutai Hou, Yongkui Lai, Cheng Chen, Wanxiang Che, Ting Liu
• Abstract summary: We propose a few-shot learning scheme that learns to bridge metric spaces of intent and slot on data-rich domains. Our model significantly outperforms the strong baselines in one and five shots settings.
• Score: 29.218780709974244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we investigate few-shot joint learning for dialogue language understanding. Most existing few-shot models learn a single task each time with only a few examples. However, dialogue language understanding contains two closely related tasks, i.e., intent detection and slot filling, and often benefits from jointly learning the two tasks. This calls for new few-shot learning techniques that are able to capture task relations from only a few examples and jointly learn multiple tasks. To achieve this, we propose a similarity-based few-shot learning scheme, named Contrastive Prototype Merging network (ConProm), that learns to bridge metric spaces of intent and slot on data-rich domains, and then adapt the bridged metric space to the specific few-shot domain. Experiments on two public datasets, Snips and FewJoint, show that our model significantly outperforms the strong baselines in one and five shots settings.

Related papers

• Leveraging knowledge distillation for partial multi-task learning from multiple remote sensing datasets [2.1178416840822023]
  Partial multi-task learning where training examples are annotated for one of the target tasks is a promising idea in remote sensing. This paper proposes using knowledge distillation to replace the need of ground truths for the alternate task and enhance the performance of such approach.
  arXiv  Detail & Related papers  (2024-05-24T09:48:50Z)
• YOLOR-Based Multi-Task Learning [12.5920336941241]
  Multi-task learning (MTL) aims to learn multiple tasks using a single model and jointly improve all of them assuming generalization and shared semantics. We propose building on You Only Learn One Representation (YOLOR), a network architecture specifically designed for multitasking. We find that our method achieves competitive performance on all tasks while maintaining a low parameter count and without any pre-training.
  arXiv  Detail & Related papers  (2023-09-29T01:42:21Z)
• Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding [52.723297744257536]
  Pre-trained language models (LMs) have shown effectiveness in scientific literature understanding tasks. We propose a multi-task contrastive learning framework, SciMult, to facilitate common knowledge sharing across different literature understanding tasks.
  arXiv  Detail & Related papers  (2023-05-23T16:47:22Z)
• A Unified Framework for Multi-intent Spoken Language Understanding with prompting [14.17726194025463]
  We describe a Prompt-based Spoken Language Understanding (PromptSLU) framework, to intuitively unify two sub-tasks into the same form. In detail, ID and SF are completed by concisely filling the utterance into task-specific prompt templates as input, and sharing output formats of key-value pairs sequence. Experiment results show that our framework outperforms several state-of-the-art baselines on two public datasets.
  arXiv  Detail & Related papers  (2022-10-07T05:58:05Z)
• Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners [67.5865966762559]
  We study whether sparsely activated Mixture-of-Experts (MoE) improve multi-task learning. We devise task-aware gating functions to route examples from different tasks to specialized experts. This results in a sparsely activated multi-task model with a large number of parameters, but with the same computational cost as that of a dense model.
  arXiv  Detail & Related papers  (2022-04-16T00:56:12Z)
• On the Efficiency of Integrating Self-supervised Learning and Meta-learning for User-defined Few-shot Keyword Spotting [51.41426141283203]
  User-defined keyword spotting is a task to detect new spoken terms defined by users. Previous works try to incorporate self-supervised learning models or apply meta-learning algorithms. Our result shows that HuBERT combined with Matching network achieves the best result.
  arXiv  Detail & Related papers  (2022-04-01T10:59:39Z)
• Low Resource Multi-Task Sequence Tagging -- Revisiting Dynamic Conditional Random Fields [67.51177964010967]
  We compare different models for low resource multi-task sequence tagging that leverage dependencies between label sequences for different tasks. We find that explicit modeling of inter-dependencies between task predictions outperforms single-task as well as standard multi-task models.
  arXiv  Detail & Related papers  (2020-05-01T07:11:34Z)
• Learning to Learn to Disambiguate: Meta-Learning for Few-Shot Word Sense Disambiguation [26.296412053816233]
  We propose a meta-learning framework for few-shot word sense disambiguation. The goal is to learn to disambiguate unseen words from only a few labeled instances. We extend several popular meta-learning approaches to this scenario, and analyze their strengths and weaknesses.
  arXiv  Detail & Related papers  (2020-04-29T17:33:31Z)
• Adversarial Continual Learning [99.56738010842301]
  We propose a hybrid continual learning framework that learns a disjoint representation for task-invariant and task-specific features. Our model combines architecture growth to prevent forgetting of task-specific skills and an experience replay approach to preserve shared skills.
  arXiv  Detail & Related papers  (2020-03-21T02:08:17Z)
• Meta-Learning across Meta-Tasks for Few-Shot Learning [107.44950540552765]
  We argue that the inter-meta-task relationships should be exploited and those tasks are sampled strategically to assist in meta-learning. We consider the relationships defined over two types of meta-task pairs and propose different strategies to exploit them.
  arXiv  Detail & Related papers  (2020-02-11T09:25:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.