Hindsight Network Credit Assignment

• URL: http://arxiv.org/abs/2011.12351v1
• Date: Tue, 24 Nov 2020 20:16:45 GMT
• Title: Hindsight Network Credit Assignment
• Authors: Kenny Young
• Abstract summary: We present Hindsight Network Credit Assignment (HNCA), a novel learning method for neural networks. HNCA works by assigning credit to each neuron's output based on how it influences the output of its immediate children in the network.
• Score: 1.370633147306388
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Hindsight Network Credit Assignment (HNCA), a novel learning method for stochastic neural networks, which works by assigning credit to each neuron's stochastic output based on how it influences the output of its immediate children in the network. We prove that HNCA provides unbiased gradient estimates while reducing variance compared to the REINFORCE estimator. We also experimentally demonstrate the advantage of HNCA over REINFORCE in a contextual bandit version of MNIST. The computational complexity of HNCA is similar to that of backpropagation. We believe that HNCA can help stimulate new ways of thinking about credit assignment in stochastic compute graphs.

Related papers

• Hamiltonian Monte Carlo on ReLU Neural Networks is Inefficient [3.823356975862005]
  We show that due to the non-differentiability of activation functions in the ReLU family, leapfrog HMC for networks with these activation functions has a large local error rate. We then verify our theoretical findings through empirical simulations as well as experiments on a real-world dataset.
  arXiv  Detail & Related papers  (2024-10-29T14:23:42Z)
• Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs [63.768739279562105]
  We show that for a particular choice of mask weights that do not depend on the learning targets, this kernel is equivalent to the NTK of the gated ReLU network on the training data. A consequence of this lack of dependence on the targets is that the NTK cannot perform better than the optimal MKL kernel on the training set.
  arXiv  Detail & Related papers  (2023-09-26T17:42:52Z)
• Layer-wise Feedback Propagation [53.00944147633484]
  We present Layer-wise Feedback Propagation (LFP), a novel training approach for neural-network-like predictors. LFP assigns rewards to individual connections based on their respective contributions to solving a given task. We demonstrate its effectiveness in achieving comparable performance to gradient descent on various models and datasets.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression [7.450181695527364]
  We study the properties of differentiable neural networks activated by rectified power unit (RePU) functions. We establish error bounds for simultaneously approximating $Cs$ smooth functions and their derivatives using RePU-activated deep neural networks.
  arXiv  Detail & Related papers  (2023-05-01T00:09:48Z)
• Can pruning improve certified robustness of neural networks? [106.03070538582222]
  We show that neural network pruning can improve empirical robustness of deep neural networks (NNs) Our experiments show that by appropriately pruning an NN, its certified accuracy can be boosted up to 8.2% under standard training. We additionally observe the existence of certified lottery tickets that can match both standard and certified robust accuracies of the original dense models.
  arXiv  Detail & Related papers  (2022-06-15T05:48:51Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Hindsight Network Credit Assignment: Efficient Credit Assignment in Networks of Discrete Stochastic Units [2.28438857884398]
  We present Hindsight Network Credit Assignment (HNCA), a novel learning algorithm for networks of discrete units. HNCA works by assigning credit to each unit based on the degree to which its output influences its immediate children in the network. We show how HNCA can be extended to optimize a more general function of the outputs of a network of units, where the function is known to the agent.
  arXiv  Detail & Related papers  (2021-10-14T20:18:38Z)
• Clustering-Based Interpretation of Deep ReLU Network [17.234442722611803]
  We recognize that the non-linear behavior of the ReLU function gives rise to a natural clustering. We propose a method to increase the level of interpretability of a fully connected feedforward ReLU neural network.
  arXiv  Detail & Related papers  (2021-10-13T09:24:11Z)
• Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks [79.74580058178594]
  We analyze the performance of training a pruned neural network by analyzing the geometric structure of the objective function. We show that the convex region near a desirable model with guaranteed generalization enlarges as the neural network model is pruned.
  arXiv  Detail & Related papers  (2021-10-12T01:11:07Z)
• Path classification by stochastic linear recurrent neural networks [2.5499055723658097]
  We show that RNNs retain a partial signature of the paths they are fed as the unique information exploited for training and classification tasks. We argue that these RNNs are easy to train and robust and back these observations with numerical experiments on both synthetic and real data.
  arXiv  Detail & Related papers  (2021-08-06T12:59:12Z)
• Local Critic Training for Model-Parallel Learning of Deep Neural Networks [94.69202357137452]
  We propose a novel model-parallel learning method, called local critic training. We show that the proposed approach successfully decouples the update process of the layer groups for both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) We also show that trained networks by the proposed method can be used for structural optimization.
  arXiv  Detail & Related papers  (2021-02-03T09:30:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.