Dendritic Neural Networks with Equilibrium Propagation

• URL: http://arxiv.org/abs/2605.08135v1
• Date: Fri, 01 May 2026 19:33:23 GMT
• Title: Dendritic Neural Networks with Equilibrium Propagation
• Abstract summary: We investigate the integration of dendritic neural networks with equilibrium propagation (EP) using an advanced EP framework.<n>Our results show that dendritic EP performance achieves comparable to standard EP on simple tasks.<n>In particular, dendritic EP significantly outperforms standard EP on KMNIST and FMNIST, and approaches the performance of dendritic networks trained with backpropagation through time.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation (BP), but its effectiveness can degrade in deeper and more challenging learning settings. In parallel, dendritic neural networks have demonstrated improved performance and generalization when trained with BP, suggesting that structured, biologically inspired architectures may enhance learning. In this work, we investigate the integration of dendritic neural networks with equilibrium propagation using an advanced EP framework. We evaluate the proposed dendritic EP model on MNIST, Kuzushiji-MNIST (KMNIST), and Fashion-MNIST (FMNIST), considering both shallow and deeper architectures. Our results show that dendritic EP achieves performance comparable to standard EP on simple tasks, while providing consistent improvements on more challenging datasets and deeper models. In particular, dendritic EP significantly outperforms standard EP on KMNIST and FMNIST, and approaches the performance of dendritic networks trained with backpropagation through time.To further understand these improvements, we analyze the evolution of hidden states during the free phase. We observe that dendritic EP exhibits higher activation magnitudes and more distributed hidden-state activity compared to standard EP, indicating that dendritic structure alters the internal network dynamics. These findings suggest that incorporating dendritic structure can enhance the effectiveness of biologically plausible learning algorithms, especially in regimes where standard EP struggles. Our work highlights the importance of architectural design for improving biologically inspired training methods.

Related papers

• General Self-Prediction Enhancement for Spiking Neurons [71.01912385372577]
  Spiking Neural Networks (SNNs) are highly energy-efficient due to event-driven, sparse computation, but their training is challenged by spike non-differentiability and trade-offs among performance, efficiency, and biological plausibility.<n>We propose a self-prediction enhanced spiking neuron method that generates an internal prediction current from its input-output history to modulate membrane potential.<n>This design offers dual advantages, it creates a continuous gradient path that alleviates vanishing gradients and boosts training stability and accuracy, while also aligning with biological principles, which resembles distal dendritic modulation and error-driven synaptic plasticity.
  arXiv  Detail & Related papers  (2026-01-29T15:08:48Z)
• StochEP: Stochastic Equilibrium Propagation for Spiking Convergent Recurrent Neural Networks [21.560035501130006]
  Spiking Networks (SNNs) promise energy-efficient, sparse, biologically inspired computation.<n>Training them with Backpropagation Through Time (BPTT) and surrogate gradients achieves strong performance but remains biologically implausible.<n>We propose an Equilibrium Propagation (EP) framework that integrates probabilistic spiking neurons into the EP paradigm.
  arXiv  Detail & Related papers  (2025-11-14T13:58:39Z)
• Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition [52.59263087086756]
  Training deep neural networks (SNNs) has critically depended on explicit normalization schemes, such as batch normalization.<n>We propose a normalization-free learning framework that incorporates lateral inhibition inspired by cortical circuits.<n>We show that our framework enables stable training of deep SNNs with biological realism and achieves competitive performance without resorting to explicit normalizations.
  arXiv  Detail & Related papers  (2025-09-27T11:11:30Z)
• Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs [17.067785532606724]
  Equilibrium Propagation (EP) is a biologically inspired local learning rule first proposed for convergent recurrent neural networks (CRNNs)<n>EP estimates gradients that closely align with those computed by Backpropagation Through Time (BPTT) while significantly reducing computational demands.<n>We propose a novel EP framework that incorporates intermediate error signals to enhance information flow and convergence of neuron dynamics.
  arXiv  Detail & Related papers  (2025-08-21T22:19:30Z)
• Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections [7.464380138405363]
  We propose a biologically plau-sible Feedback-regulated REsidual recurrent neural network (FRE-RNN) to study its learning performance.<n>The improvement in con-vergence property reduces the computational cost and train-ing time of EP by orders of magnitude.<n>Our approach substantially enhances the applicabil-ity and practicality of EP in large-scale networks that un-derpin artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-05T15:07:50Z)
• Developmental Plasticity-inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networks [11.730984231143108]
  Developmental plasticity plays prominent role in shaping the brain's structure during ongoing learning. Existing network compression methods for deep artificial neural networks (ANNs) and spiking neural networks (SNNs) draw little inspiration from brain's developmental plasticity mechanisms. This paper proposes a developmental plasticity-inspired adaptive pruning (DPAP) method, with inspiration from the adaptive developmental pruning of dendritic spines, synapses, and neurons.
  arXiv  Detail & Related papers  (2022-11-23T05:26:51Z)
• On the Intrinsic Structures of Spiking Neural Networks [66.57589494713515]
  Recent years have emerged a surge of interest in SNNs owing to their remarkable potential to handle time-dependent and event-driven data. There has been a dearth of comprehensive studies examining the impact of intrinsic structures within spiking computations. This work delves deep into the intrinsic structures of SNNs, by elucidating their influence on the expressivity of SNNs.
  arXiv  Detail & Related papers  (2022-06-21T09:42:30Z)
• MAP-SNN: Mapping Spike Activities with Multiplicity, Adaptability, and Plasticity into Bio-Plausible Spiking Neural Networks [4.806663076114504]
  Spiking Neural Network (SNN) is considered more biologically realistic and power-efficient as it imitates the fundamental mechanism of the human brain. We consider three properties in modeling spike activities: Multiplicity, Adaptability, and Plasticity (MAP) The proposed SNN model achieves competitive performances on neuromorphic datasets: N-MNIST and SHD.
  arXiv  Detail & Related papers  (2022-04-21T05:36:11Z)
• Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias [62.43908463620527]
  In practice, EP does not scale to visual tasks harder than MNIST. We show that a bias in the gradient estimate of EP, inherent in the use of finite nudging, is responsible for this phenomenon. These results highlight EP as a scalable approach to compute error gradients in deep neural networks, thereby motivating its hardware implementation.
  arXiv  Detail & Related papers  (2021-01-14T10:23:40Z)
• A Theoretical Framework for Target Propagation [75.52598682467817]
  We analyze target propagation (TP), a popular but not yet fully understood alternative to backpropagation (BP) Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training.
  arXiv  Detail & Related papers  (2020-06-25T12:07:06Z)
• Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation [69.87491240509485]
  Equilibrium Propagation (EP) is a biologically inspired alternative algorithm to backpropagation (BP) for training neural networks. We propose a discrete-time formulation of EP which enables to simplify equations, speed up training and extend EP to CNNs. Our CNN model achieves the best performance ever reported on MNIST with EP.
  arXiv  Detail & Related papers  (2020-04-29T12:14:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.