MPLP: Learning a Message Passing Learning Protocol

• URL: http://arxiv.org/abs/2007.00970v2
• Date: Fri, 3 Jul 2020 10:28:35 GMT
• Title: MPLP: Learning a Message Passing Learning Protocol
• Authors: Ettore Randazzo, Eyvind Niklasson, Alexander Mordvintsev
• Abstract summary: We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP) We abstract every operations occurring in ANNs as independent agents. Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents.
• Score: 63.948465205530916
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP). In MPLP, we abstract every operations occurring in ANNs as independent agents. Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents. We demonstrate the viability of MPLP as opposed to traditional gradient-based approaches on simple feed-forward neural networks, and present a framework capable of generalizing to non-traditional neural network architectures. MPLP is meta learned using end-to-end gradient-based meta-optimisation. We further discuss the observed properties of MPLP and hypothesize its applicability on various fields of deep learning.

Related papers

• Vision-Language Models Provide Promptable Representations for Reinforcement Learning [67.40524195671479]
  We propose a novel approach that uses the vast amounts of general and indexable world knowledge encoded in vision-language models (VLMs) pre-trained on Internet-scale data for embodied reinforcement learning (RL) We show that our approach can use chain-of-thought prompting to produce representations of common-sense semantic reasoning, improving policy performance in novel scenes by 1.5 times.
  arXiv  Detail & Related papers  (2024-02-05T00:48:56Z)
• Neural Routing in Meta Learning [9.070747377130472]
  We aim to improve the model performance of the current meta learning algorithms by selectively using only parts of the model conditioned on the input tasks. In this work, we describe an approach that investigates task-dependent dynamic neuron selection in deep convolutional neural networks (CNNs) by leveraging the scaling factor in the batch normalization layer. We find that the proposed approach, neural routing in meta learning (NRML), outperforms one of the well-known existing meta learning baselines on few-shot classification tasks.
  arXiv  Detail & Related papers  (2022-10-14T16:31:24Z)
• Meta Learning for Natural Language Processing: A Survey [88.58260839196019]
  Deep learning has been the mainstream technique in natural language processing (NLP) area. Deep learning requires many labeled data and is less generalizable across domains. Meta-learning is an arising field in machine learning studying approaches to learn better algorithms.
  arXiv  Detail & Related papers  (2022-05-03T13:58:38Z)
• Learning Meta Pattern for Face Anti-Spoofing [26.82129880310214]
  Face Anti-Spoofing (FAS) is essential to secure face recognition systems. Recent hybrid methods have been explored to extract task-aware handcrafted features. We propose a learnable network to extract Meta Pattern (MP) in our learning-to-learn framework.
  arXiv  Detail & Related papers  (2021-10-13T14:34:20Z)
• FedNLP: A Research Platform for Federated Learning in Natural Language Processing [55.01246123092445]
  We present the FedNLP, a research platform for federated learning in NLP. FedNLP supports various popular task formulations in NLP such as text classification, sequence tagging, question answering, seq2seq generation, and language modeling. Preliminary experiments with FedNLP reveal that there exists a large performance gap between learning on decentralized and centralized datasets.
  arXiv  Detail & Related papers  (2021-04-18T11:04:49Z)
• Few-shot Learning for Slot Tagging with Attentive Relational Network [35.624877181332636]
  Metric-based learning is a well-known family of methods for few-shot learning, especially in computer vision. We propose a novel metric-based learning architecture - Attentive Network. The results on SNIPS show that our proposed method outperforms other state-of-the-art metric-based learning methods.
  arXiv  Detail & Related papers  (2021-03-03T11:24:24Z)
• Meta-Learning for Natural Language Understanding under Continual Learning Framework [2.620911206953405]
  We implement the model-agnostic meta-learning (MAML) and Online aware Meta-learning (OML) meta-objective under the continual framework for NLU tasks. We validate our methods on selected SuperGLUE and GLUE benchmark.
  arXiv  Detail & Related papers  (2020-11-03T03:41:10Z)
• Neural Function Modules with Sparse Arguments: A Dynamic Approach to Integrating Information across Layers [84.57980167400513]
  Neural Function Modules (NFM) aims to introduce the same structural capability into deep learning. Most of the work in the context of feed-forward networks combining top-down and bottom-up feedback is limited to classification problems. The key contribution of our work is to combine attention, sparsity, top-down and bottom-up feedback, in a flexible algorithm.
  arXiv  Detail & Related papers  (2020-10-15T20:43:17Z)
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer [64.22926988297685]
  Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP) In this paper, we explore the landscape of introducing transfer learning techniques for NLP by a unified framework that converts all text-based language problems into a text-to-text format.
  arXiv  Detail & Related papers  (2019-10-23T17:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.