Closed-form predictive coding via hierarchical Gaussian filters

• URL: http://arxiv.org/abs/2605.20293v1
• Date: Tue, 19 May 2026 10:11:41 GMT
• Title: Closed-form predictive coding via hierarchical Gaussian filters
• Abstract summary: Predictive coding offers a local and biologically grounded alternative to backpropagation in the training of artificial neural networks.<n>On FashionMNIST, our solution approaches backpropagation in epoch-level wall-clock cost while converging in fewer epochs.<n>We establish that closed-form variational inference with online precision learning provides a tractable foundation for deep predictive coding networks.
• Score: 33.92034506804344
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predictive coding (PC) offers a local and biologically grounded alternative to backpropagation in the training of artificial neural networks, yet to date, it remains slower, and performance degrades sharply as network depth increases. We trace both problems to a single simplification: current PC networks fix the precision matrix to the identity, discarding precision-weighted prediction errors that the variational derivation requires to be fast, local, and Bayesian. We close this gap by expressing predictive coding networks as deep hierarchical Gaussian filters (HGFs) and restore precision-weighted message passing, yielding dynamic uncertainty estimates and Hebbian-compatible update rules at every layer. The resulting networks can simultaneously learn activations, weights, and precisions under a single free-energy objective, with no global error signal, and resolve inference without requiring iterations or automatic differentiation. On FashionMNIST, our solution approaches backpropagation in epoch-level wall-clock cost while converging in fewer epochs, and outperforms it on online, data efficiency, and concept-drift tasks. We thus establish that closed-form variational inference with online precision learning provides a tractable foundation for deep predictive coding networks, retaining biological and interpretative advantages, without requiring iterative relaxation or global error signals.

Related papers

• Predictive Coding-based Deep Neural Network Fine-tuning for Computationally Efficient Domain Adaptation [5.013248430919224]
  We propose a hybrid training methodology that enables efficient on-device domain adaptation.<n>The method begins with a deep neural network trained offline using Backpropagation to achieve high initial performance.<n> Predictive Coding is employed for online adaptation, allowing the model to recover accuracy lost due to shifts in the input data distribution.
  arXiv  Detail & Related papers  (2025-09-24T16:03:27Z)
• Neural Conformal Control for Time Series Forecasting [54.96087475179419]
  We introduce a neural network conformal prediction method for time series that enhances adaptivity in non-stationary environments.<n>Our approach acts as a neural controller designed to achieve desired target coverage, leveraging auxiliary multi-view data with neural network encoders.<n>We empirically demonstrate significant improvements in coverage and probabilistic accuracy, and find that our method is the only one that combines good calibration with consistency in prediction intervals.
  arXiv  Detail & Related papers  (2024-12-24T03:56:25Z)
• Deep Neural Networks Tend To Extrapolate Predictably [51.303814412294514]
  neural network predictions tend to be unpredictable and overconfident when faced with out-of-distribution (OOD) inputs. We observe that neural network predictions often tend towards a constant value as input data becomes increasingly OOD. We show how one can leverage our insights in practice to enable risk-sensitive decision-making in the presence of OOD inputs.
  arXiv  Detail & Related papers  (2023-10-02T03:25:32Z)
• Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding [57.71603937699949]
  We study optimization guarantees, i.e., achieving near-zero training loss with the increase in the number of learning epochs. We show that the threshold on the number of training samples increases with the increase in the network width.
  arXiv  Detail & Related papers  (2023-09-12T13:03:47Z)
• Learning k-Level Structured Sparse Neural Networks Using Group Envelope Regularization [4.0554893636822]
  We introduce a novel approach to deploy large-scale Deep Neural Networks on constrained resources. The method speeds up inference time and aims to reduce memory demand and power consumption.
  arXiv  Detail & Related papers  (2022-12-25T15:40:05Z)
• CorrectNet: Robustness Enhancement of Analog In-Memory Computing for Neural Networks by Error Suppression and Compensation [4.570841222958966]
  We propose a framework to enhance the robustness of neural networks under variations and noise. We show that inference accuracy of neural networks can be recovered from as low as 1.69% under variations and noise.
  arXiv  Detail & Related papers  (2022-11-27T19:13:33Z)
• Predictive coding, precision and natural gradients [2.1601966913620325]
  We show that hierarchical predictive coding networks with learnable precision are able to solve various supervised and unsupervised learning tasks. When applied to unsupervised auto-encoding of image inputs, the deterministic network produces hierarchically organized and disentangled embeddings.
  arXiv  Detail & Related papers  (2021-11-12T21:05:03Z)
• Performance and accuracy assessments of an incompressible fluid solver coupled with a deep Convolutional Neural Network [0.0]
  The resolution of the Poisson equation is usually one of the most computationally intensive steps for incompressible fluid solvers. CNN has been introduced to solve this equation, leading to significant inference time reduction. A hybrid strategy is developed, which couples a CNN with a traditional iterative solver to ensure a user-defined accuracy level.
  arXiv  Detail & Related papers  (2021-09-20T08:30:29Z)
• Adaptive Anomaly Detection for Internet of Things in Hierarchical Edge Computing: A Contextual-Bandit Approach [81.5261621619557]
  We propose an adaptive anomaly detection scheme with hierarchical edge computing (HEC) We first construct multiple anomaly detection DNN models with increasing complexity, and associate each of them to a corresponding HEC layer. Then, we design an adaptive model selection scheme that is formulated as a contextual-bandit problem and solved by using a reinforcement learning policy network.
  arXiv  Detail & Related papers  (2021-08-09T08:45:47Z)
• Non-Gradient Manifold Neural Network [79.44066256794187]
  Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent. We propose a novel manifold neural network based on non-gradient optimization.
  arXiv  Detail & Related papers  (2021-06-15T06:39:13Z)
• Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming [97.40955121478716]
  We propose a first-order dual SDP algorithm that requires memory only linear in the total number of network activations. We significantly improve L-inf verified robust accuracy from 1% to 88% and 6% to 40% respectively. We also demonstrate tight verification of a quadratic stability specification for the decoder of a variational autoencoder.
  arXiv  Detail & Related papers  (2020-10-22T12:32:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.