Few-shot Learning for Slot Tagging with Attentive Relational Network
- URL: http://arxiv.org/abs/2103.02333v1
- Date: Wed, 3 Mar 2021 11:24:24 GMT
- Title: Few-shot Learning for Slot Tagging with Attentive Relational Network
- Authors: Cennet Oguz, Ngoc Thang Vu
- Abstract summary: Metric-based learning is a well-known family of methods for few-shot learning, especially in computer vision.
We propose a novel metric-based learning architecture - Attentive Network.
The results on SNIPS show that our proposed method outperforms other state-of-the-art metric-based learning methods.
- Score: 35.624877181332636
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Metric-based learning is a well-known family of methods for few-shot
learning, especially in computer vision. Recently, they have been used in many
natural language processing applications but not for slot tagging. In this
paper, we explore metric-based learning methods in the slot tagging task and
propose a novel metric-based learning architecture - Attentive Relational
Network. Our proposed method extends relation networks, making them more
suitable for natural language processing applications in general, by leveraging
pretrained contextual embeddings such as ELMO and BERT and by using attention
mechanism. The results on SNIPS data show that our proposed method outperforms
other state-of-the-art metric-based learning methods.
Related papers
- Semantic Meta-Split Learning: A TinyML Scheme for Few-Shot Wireless Image Classification [50.28867343337997]
This work presents a TinyML-based semantic communication framework for few-shot wireless image classification.
We exploit split-learning to limit the computations performed by the end-users while ensuring privacy-preserving.
meta-learning overcomes data availability concerns and speeds up training by utilizing similarly trained tasks.
arXiv Detail & Related papers (2024-09-03T05:56:55Z) - Language-guided Skill Learning with Temporal Variational Inference [38.733622157088035]
We present an algorithm for skill discovery from expert demonstrations.
Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning.
arXiv Detail & Related papers (2024-02-26T07:19:23Z) - Automating Knowledge Acquisition for Content-Centric Cognitive Agents
Using LLMs [0.0]
The paper describes a system that uses large language model (LLM) technology to support the automatic learning of new entries in an intelligent agent's semantic lexicon.
The process is bootstrapped by an existing non-toy lexicon and a natural language generator that converts formal, ontologically-grounded representations of meaning into natural language sentences.
arXiv Detail & Related papers (2023-12-27T02:31:51Z) - Learning Symbolic Rules over Abstract Meaning Representations for
Textual Reinforcement Learning [63.148199057487226]
We propose a modular, NEuroSymbolic Textual Agent (NESTA) that combines a generic semantic generalization with a rule induction system to learn interpretable rules as policies.
Our experiments show that the proposed NESTA method outperforms deep reinforcement learning-based techniques by achieving better to unseen test games and learning from fewer training interactions.
arXiv Detail & Related papers (2023-07-05T23:21:05Z) - Incremental Embedding Learning via Zero-Shot Translation [65.94349068508863]
Current state-of-the-art incremental learning methods tackle catastrophic forgetting problem in traditional classification networks.
We propose a novel class-incremental method for embedding network, named as zero-shot translation class-incremental method (ZSTCI)
In addition, ZSTCI can easily be combined with existing regularization-based incremental learning methods to further improve performance of embedding networks.
arXiv Detail & Related papers (2020-12-31T08:21:37Z) - Systematic Generalization on gSCAN with Language Conditioned Embedding [19.39687991647301]
Systematic Generalization refers to a learning algorithm's ability to extrapolate learned behavior to unseen situations.
We propose a novel method that learns objects' contextualized embeddings with dynamic message passing conditioned on the input natural language.
arXiv Detail & Related papers (2020-09-11T17:35:05Z) - Region Comparison Network for Interpretable Few-shot Image
Classification [97.97902360117368]
Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes.
We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works.
We also present a new way to generalize the interpretability from the level of tasks to categories.
arXiv Detail & Related papers (2020-09-08T07:29:05Z) - MPLP: Learning a Message Passing Learning Protocol [63.948465205530916]
We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP)
We abstract every operations occurring in ANNs as independent agents.
Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents.
arXiv Detail & Related papers (2020-07-02T09:03:14Z) - Modeling Document Interactions for Learning to Rank with Regularized
Self-Attention [22.140197412459393]
We explore modeling documents interactions with self-attention based neural networks.
We propose simple yet effective regularization terms designed to model interactions between documents.
We show that training self-attention network with our proposed regularization terms can significantly outperform existing learning to rank methods.
arXiv Detail & Related papers (2020-05-08T09:53:31Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.