A Neural Network Training Method Based on Neuron Connection Coefficient Adjustments

• URL: http://arxiv.org/abs/2502.10414v1
• Date: Sat, 25 Jan 2025 16:07:30 GMT
• Title: A Neural Network Training Method Based on Neuron Connection Coefficient Adjustments
• Authors: Kun Jiang,
• Abstract summary: We present a new training approach for a neural network based on symmetric differential equations.<n>Unlike the previously introduced method, this approach does not require adjustments to the fixed points of the differential equations.<n>To validate this approach, we tested it on the MNIST dataset and achieved promising results.
• Score: 2.684545081600664
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In previous studies, we introduced a neural network framework based on symmetric differential equations, along with one of its training methods. In this article, we present another training approach for this neural network. This method leverages backward signal propagation and eliminates reliance on the traditional chain derivative rule, offering a high degree of biological interpretability. Unlike the previously introduced method, this approach does not require adjustments to the fixed points of the differential equations. Instead, it focuses solely on modifying the connection coefficients between neurons, closely resembling the training process of traditional multilayer perceptron (MLP) networks. By adopting a suitable adjustment strategy, this method effectively avoids certain potential local minima. To validate this approach, we tested it on the MNIST dataset and achieved promising results. Through further analysis, we identified certain limitations of the current neural network architecture and proposed measures for improvement.

Related papers

• A Neural Network Training Method Based on Distributed PID Control [0.0]
  In the previous article, we introduced a neural network framework based on symmetric differential equations. This study proposes an alternative training approach that utilizes differential equation signal propagation instead of chain rule derivation.
  arXiv  Detail & Related papers  (2024-11-18T19:25:26Z)
• Randomized Forward Mode Gradient for Spiking Neural Networks in Scientific Machine Learning [4.178826560825283]
  Spiking neural networks (SNNs) represent a promising approach in machine learning, combining the hierarchical learning capabilities of deep neural networks with the energy efficiency of spike-based computations. Traditional end-to-end training of SNNs is often based on back-propagation, where weight updates are derived from gradients computed through the chain rule. This method encounters challenges due to its limited biological plausibility and inefficiencies on neuromorphic hardware. In this study, we introduce an alternative training approach for SNNs. Instead of using back-propagation, we leverage weight perturbation methods within a forward-mode
  arXiv  Detail & Related papers  (2024-11-11T15:20:54Z)
• Learning by the F-adjoint [0.0]
  In this work, we develop and investigate this theoretical framework to improve some supervised learning algorithm for feed-forward neural network. Our main result is that by introducing some neural dynamical model combined by the gradient descent algorithm, we derived an equilibrium F-adjoint process. Experimental results on MNIST and Fashion-MNIST datasets, demonstrate that the proposed approach provide a significant improvements on the standard back-propagation training procedure.
  arXiv  Detail & Related papers  (2024-07-08T13:49:25Z)
• Chebyshev Spectral Neural Networks for Solving Partial Differential Equations [0.0]
  The study uses a feedforward neural network model and error backpropagation principles, utilizing automatic differentiation (AD) to compute the loss function. The numerical efficiency and accuracy of the CSNN model are investigated through testing on elliptic partial differential equations, and it is compared with the well-known Physics-Informed Neural Network(PINN) method.
  arXiv  Detail & Related papers  (2024-06-06T05:31:45Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Scalable computation of prediction intervals for neural networks via matrix sketching [79.44177623781043]
  Existing algorithms for uncertainty estimation require modifying the model architecture and training procedure. This work proposes a new algorithm that can be applied to a given trained neural network and produces approximate prediction intervals.
  arXiv  Detail & Related papers  (2022-05-06T13:18:31Z)
• An alternative approach to train neural networks using monotone variational inequality [22.320632565424745]
  We propose an alternative approach to neural network training using the monotone vector field. Our approach can be used for more efficient fine-tuning of a pre-trained neural network.
  arXiv  Detail & Related papers  (2022-02-17T19:24:20Z)
• LocalDrop: A Hybrid Regularization for Deep Neural Networks [98.30782118441158]
  We propose a new approach for the regularization of neural networks by the local Rademacher complexity called LocalDrop. A new regularization function for both fully-connected networks (FCNs) and convolutional neural networks (CNNs) has been developed based on the proposed upper bound of the local Rademacher complexity.
  arXiv  Detail & Related papers  (2021-03-01T03:10:11Z)
• Developing Constrained Neural Units Over Time [81.19349325749037]
  This paper focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. The structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner.
  arXiv  Detail & Related papers  (2020-09-01T09:07:25Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• MSE-Optimal Neural Network Initialization via Layer Fusion [68.72356718879428]
  Deep neural networks achieve state-of-the-art performance for a range of classification and inference tasks. The use of gradient combined nonvolutionity renders learning susceptible to novel problems. We propose fusing neighboring layers of deeper networks that are trained with random variables.
  arXiv  Detail & Related papers  (2020-01-28T18:25:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.