Unbiased Weight Maximization

• URL: http://arxiv.org/abs/2307.13270v1
• Date: Tue, 25 Jul 2023 05:45:52 GMT
• Title: Unbiased Weight Maximization
• Authors: Stephen Chung
• Abstract summary: We propose a new learning rule for a network of Bernoulli-logistic units that is unbiased and scales well with the number of network's units in terms of learning speed. Notably, to our knowledge, this is the first learning rule for a network of Bernoulli-logistic units that is unbiased and scales well with the number of network's units in terms of learning speed.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A biologically plausible method for training an Artificial Neural Network (ANN) involves treating each unit as a stochastic Reinforcement Learning (RL) agent, thereby considering the network as a team of agents. Consequently, all units can learn via REINFORCE, a local learning rule modulated by a global reward signal, which aligns more closely with biologically observed forms of synaptic plasticity. Nevertheless, this learning method is often slow and scales poorly with network size due to inefficient structural credit assignment, since a single reward signal is broadcast to all units without considering individual contributions. Weight Maximization, a proposed solution, replaces a unit's reward signal with the norm of its outgoing weight, thereby allowing each hidden unit to maximize the norm of the outgoing weight instead of the global reward signal. In this research report, we analyze the theoretical properties of Weight Maximization and propose a variant, Unbiased Weight Maximization. This new approach provides an unbiased learning rule that increases learning speed and improves asymptotic performance. Notably, to our knowledge, this is the first learning rule for a network of Bernoulli-logistic units that is unbiased and scales well with the number of network's units in terms of learning speed.

Related papers

• Dense Reward for Free in Reinforcement Learning from Human Feedback [64.92448888346125]
  We leverage the fact that the reward model contains more information than just its scalar output. We use these attention weights to redistribute the reward along the whole completion. Empirically, we show that it stabilises training, accelerates the rate of learning, and, in practical cases, may lead to better local optima.
  arXiv  Detail & Related papers  (2024-02-01T17:10:35Z)
• FedNAR: Federated Optimization with Normalized Annealing Regularization [54.42032094044368]
  We explore the choices of weight decay and identify that weight decay value appreciably influences the convergence of existing FL algorithms. We develop Federated optimization with Normalized Annealing Regularization (FedNAR), a plug-in that can be seamlessly integrated into any existing FL algorithms.
  arXiv  Detail & Related papers  (2023-10-04T21:11:40Z)
• Structural Credit Assignment with Coordinated Exploration [0.0]
  Methods aimed at improving structural credit assignment can generally be classified into two categories. We propose the use of Boltzmann machines or a recurrent network for coordinated exploration. Experimental results demonstrate that coordinated exploration significantly exceeds independent exploration in training speed.
  arXiv  Detail & Related papers  (2023-07-25T04:55:45Z)
• IF2Net: Innately Forgetting-Free Networks for Continual Learning [49.57495829364827]
  Continual learning can incrementally absorb new concepts without interfering with previously learned knowledge. Motivated by the characteristics of neural networks, we investigated how to design an Innately Forgetting-Free Network (IF2Net) IF2Net allows a single network to inherently learn unlimited mapping rules without telling task identities at test time.
  arXiv  Detail & Related papers  (2023-06-18T05:26:49Z)
• NN-EUCLID: deep-learning hyperelasticity without stress data [0.0]
  We propose a new approach for unsupervised learning of hyperelastic laws with physics-consistent deep neural networks. In contrast to supervised learning, which assumes the stress-strain, the approach only uses realistically measurable full-elastic field displacement and global force availability data.
  arXiv  Detail & Related papers  (2022-05-04T13:54:54Z)
• Learning by Competition of Self-Interested Reinforcement Learning Agents [0.0]
  An artificial neural network can be trained by uniformly broadcasting a reward signal to units that implement a REINFORCE learning rule. We propose replacing the reward signal to hidden units with the change in the $L2$ norm of the unit's outgoing weight. Our experiments show that a network trained with Weight Maximization can learn significantly faster than REINFORCE and slightly slower than backpropagation.
  arXiv  Detail & Related papers  (2020-10-19T18:18:53Z)
• MAP Propagation Algorithm: Faster Learning with a Team of Reinforcement Learning Agents [0.0]
  An alternative way of training an artificial neural network is through treating each unit in the network as a reinforcement learning agent. We propose a novel algorithm called MAP propagation to reduce this variance significantly. Our work thus allows for the broader application of the teams of agents in deep reinforcement learning.
  arXiv  Detail & Related papers  (2020-10-15T17:17:39Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)
• Neural networks with late-phase weights [66.72777753269658]
  We show that the solutions found by SGD can be further improved by ensembling a subset of the weights in late stages of learning. At the end of learning, we obtain back a single model by taking a spatial average in weight space.
  arXiv  Detail & Related papers  (2020-07-25T13:23:37Z)
• Distance-Based Regularisation of Deep Networks for Fine-Tuning [116.71288796019809]
  We develop an algorithm that constrains a hypothesis class to a small sphere centred on the initial pre-trained weights. Empirical evaluation shows that our algorithm works well, corroborating our theoretical results.
  arXiv  Detail & Related papers  (2020-02-19T16:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.