A Basic Evaluation of Neural Networks Trained with the Error Diffusion Learning Algorithm

• URL: http://arxiv.org/abs/2504.14814v2
• Date: Mon, 28 Apr 2025 11:14:08 GMT
• Title: A Basic Evaluation of Neural Networks Trained with the Error Diffusion Learning Algorithm
• Authors: Kazuhisa Fujita,
• Abstract summary: Kaneko's Error Diffusion Learning Algorithm (EDLA)<n>A single global error signal diffuses throughout a network composed of paired excitatory-inhibitory sublayers.<n>Experiments indicate that EDLA networks can consistently achieve high accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artificial neural networks are powerful tools capable of addressing various tasks. Although the backpropagation algorithm has become a standard training method for these neural networks, its lack of biological plausibility has inspired the development of alternative learning approaches. One such alternative is Kaneko's Error Diffusion Learning Algorithm (EDLA), a biologically motivated approach wherein a single global error signal diffuses throughout a network composed of paired excitatory-inhibitory sublayers, thereby eliminating the necessity for layer-wise backpropagation. This study presents a contemporary formulation of the EDLA framework and evaluates its effectiveness through parity check, regression, and image classification tasks. Our experimental results indicate that EDLA networks can consistently achieve high accuracy across these benchmarks, with performance efficiency and convergence speed notably influenced by the choice of learning rate, neuron count, and network depth. Further investigation of the internal representations formed by EDLA networks reveals their capacity for meaningful feature extraction, similar to traditional neural networks. These results suggest that EDLA is a biologically motivated alternative for training feedforward networks and will motivate future work on extending this method to biologically inspired neural networks.

Related papers

• Efficient Machine Unlearning via Influence Approximation [75.31015485113993]
  Influence-based unlearning has emerged as a prominent approach to estimate the impact of individual training samples on model parameters without retraining.<n>This paper establishes a theoretical link between memorizing (incremental learning) and forgetting (unlearning)<n>We introduce the Influence Approximation Unlearning algorithm for efficient machine unlearning from the incremental perspective.
  arXiv  Detail & Related papers  (2025-07-31T05:34:27Z)
• Graph Neural Network-Based Reinforcement Learning for Controlling Biological Networks: The GATTACA Framework [0.0]
  We explore the use of deep reinforcement learning (DRL) to control network models of complex biological systems.<n>We formulate a novel control problem for Boolean network models under the asynchronous update mode in the context of cellular reprogramming.<n>To leverage the structure of biological systems, we incorporate graph neural networks with graph convolutions into the artificial neural network approximator for the action-value function learned by the DRL agent.
  arXiv  Detail & Related papers  (2025-05-05T15:07:20Z)
• A new training approach for text classification in Mental Health: LatentGLoss [0.0]
  This study presents a multi-stage approach to mental health classification by leveraging machine learning algorithms, deep learning architectures, and transformer-based models.<n>The core contribution of this study lies in a novel training strategy involving a dual-model architecture composed of a teacher and a student network.<n>The experimental results highlight the effectiveness of each modeling stage and demonstrate that the proposed loss function and teacher-student interaction significantly enhance the model's learning capacity in mental health prediction tasks.
  arXiv  Detail & Related papers  (2025-04-09T19:34:31Z)
• Discovering Chunks in Neural Embeddings for Interpretability [53.80157905839065]
  We propose leveraging the principle of chunking to interpret artificial neural population activities.<n>We first demonstrate this concept in recurrent neural networks (RNNs) trained on artificial sequences with imposed regularities.<n>We identify similar recurring embedding states corresponding to concepts in the input, with perturbations to these states activating or inhibiting the associated concepts.
  arXiv  Detail & Related papers  (2025-02-03T20:30:46Z)
• Feasibility Analysis of Federated Neural Networks for Explainable Detection of Atrial Fibrillation [1.6053176639259055]
  Early detection of atrial fibrillation (AFib) is challenging due to its asymptomatic and paroxysmal nature. This study assesses the feasibility of training a neural network on a Federated Learning (FL) platform to detect AFib using raw ECG data.
  arXiv  Detail & Related papers  (2024-10-14T15:06:10Z)
• Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning [32.122425860826525]
  error backpropagation has faced criticism for its lack of biological plausibility. We propose counter-current learning (CCL), a biologically plausible framework for credit assignment in neural networks. Our work presents a direction for biologically inspired and plausible learning algorithms, offering an alternative mechanism of learning and adaptation in neural networks.
  arXiv  Detail & Related papers  (2024-09-30T00:47:13Z)
• Enhancing Cognitive Workload Classification Using Integrated LSTM Layers and CNNs for fNIRS Data Analysis [13.74551296919155]
  This paper explores the im-pact of Long Short-Term Memory layers on the effectiveness of Convolutional Neural Networks (CNNs) within deep learning models. By integrating LSTM layers, the model can capture temporal dependencies in the fNIRS data, al-lowing for a more comprehensive understanding of cognitive states.
  arXiv  Detail & Related papers  (2024-07-22T11:28:34Z)
• Emerging NeoHebbian Dynamics in Forward-Forward Learning: Implications for Neuromorphic Computing [7.345136916791223]
  Forward-Forward Algorithm (FFA) employs local learning rules for each layer. We show that when employing a squared Euclidean norm as a goodness function driving the local learning, the resulting FFA is equivalent to a neo-Hebbian Learning Rule.
  arXiv  Detail & Related papers  (2024-06-24T09:33:56Z)
• Sparse Multitask Learning for Efficient Neural Representation of Motor Imagery and Execution [30.186917337606477]
  We introduce a sparse multitask learning framework for motor imagery (MI) and motor execution (ME) tasks. Given a dual-task CNN model for MI-ME classification, we apply a saliency-based sparsification approach to prune superfluous connections. Our results indicate that this tailored sparsity can mitigate the overfitting problem and improve the test performance with small amount of data.
  arXiv  Detail & Related papers  (2023-12-10T09:06:16Z)
• Spiking Generative Adversarial Network with Attention Scoring Decoding [4.5727987473456055]
  Spiking neural networks offer a closer approximation to brain-like processing. We build a spiking generative adversarial network capable of handling complex images.
  arXiv  Detail & Related papers  (2023-05-17T14:35:45Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
  We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe. GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
  arXiv  Detail & Related papers  (2023-02-06T16:38:43Z)
• Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Network [32.575847142016585]
  Top-Down Credit Assignment Network (TDCA-network) is designed to train a bottom-up network using a Top-Down Credit Assignment Network (TDCA-network) TDCA-network serves as a substitute for the conventional loss function and the back-propagation algorithm, widely used in neural network training. The results indicate TDCA-network holds promising potential to train neural networks across diverse datasets.
  arXiv  Detail & Related papers  (2022-08-01T07:14:37Z)
• Initial Study into Application of Feature Density and Linguistically-backed Embedding to Improve Machine Learning-based Cyberbullying Detection [54.83707803301847]
  The research was conducted on a Formspring dataset provided in a Kaggle competition on automatic cyberbullying detection. The study confirmed the effectiveness of Neural Networks in cyberbullying detection and the correlation between classifier performance and Feature Density.
  arXiv  Detail & Related papers  (2022-06-04T03:17:15Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Improving Music Performance Assessment with Contrastive Learning [78.8942067357231]
  This study investigates contrastive learning as a potential method to improve existing MPA systems. We introduce a weighted contrastive loss suitable for regression tasks applied to a convolutional neural network. Our results show that contrastive-based methods are able to match and exceed SoTA performance for MPA regression tasks.
  arXiv  Detail & Related papers  (2021-08-03T19:24:25Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)
• Efficient Estimation of Influence of a Training Instance [56.29080605123304]
  We propose an efficient method for estimating the influence of a training instance on a neural network model. Our method is inspired by dropout, which zero-masks a sub-network and prevents the sub-network from learning each training instance. We demonstrate that the proposed method can capture training influences, enhance the interpretability of error predictions, and cleanse the training dataset for improving generalization.
  arXiv  Detail & Related papers  (2020-12-08T04:31:38Z)
• Network Diffusions via Neural Mean-Field Dynamics [52.091487866968286]
  We propose a novel learning framework for inference and estimation problems of diffusion on networks. Our framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities. Our approach is versatile and robust to variations of the underlying diffusion network models.
  arXiv  Detail & Related papers  (2020-06-16T18:45:20Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.