Augmented Lagrangian Predictive Coding

• URL: http://arxiv.org/abs/2605.31022v1
• Date: Fri, 29 May 2026 08:54:21 GMT
• Title: Augmented Lagrangian Predictive Coding
• Abstract summary: Predictive coding (PC) is a local-learning alternative to backpropagation (BP)<n>We introduce Augmented Lagrangian Predictive Coding (PC-ALM)<n>We analyze PC-ALM in nonlinear PC networks up to depth 128.
• Score: 0.7980273012483662
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predictive coding (PC) is a local-learning alternative to backpropagation (BP), training deep networks via local energy-minimization dynamics rather than a global backward pass. We introduce Augmented Lagrangian Predictive Coding (PC-ALM), which maintains PC's inference budget but aligns each weight update toward BP by accumulating per-layer constraint errors into a layer-local Lagrange multiplier. In linear PC networks, PC-ALM converges to an equilibrium with exact BP gradients distributed across the network via only layer-local updates. We analyze PC-ALM in nonlinear PC networks up to depth 128 and show that it matches BP performance across all width-depth regimes, notably in deep narrow networks where PC underperforms. PC-ALM introduces recurrent dynamics in each layer's activations. Compared to PC's heat flow on a scalar energy, PC-ALM dynamics are driven by dual ascent on the augmented Lagrangian. We observe "ballistic" credit propagation across very deep networks, with credit signals evenly distributed across layers, compared to PC's slow, diffusive credit propagation. Beyond the algorithm itself, the augmented Lagrangian framework offers a generalization of PC, and may yield insights into how distributed systems could compute and propagate BP-like credit signals through purely local dynamics.

Related papers

• Rethinking the Divergence Regularization in LLM RL [56.952725733148746]
  Reinforcement learning (RL) has become a key component of post-training large language models (LLMs)<n>We propose Divergence Regularized Policy Optimization (DRPO), which replaces the hard mask with a smooth advantage-weighted quadratic regularizer on policy shift.<n>DRPO preserves the same trust-region geometry as DPPO while inducing bounded, continuous gradient weights.
  arXiv  Detail & Related papers  (2026-06-08T17:58:23Z)
• Accelerated Predictive Coding Networks via Direct Kolen-Pollack Feedback Alignment [7.328567184271344]
  Predictive coding (PC) is a biologically inspired algorithm for training neural networks that relies only on local updates.<n>We propose direct Kolen-Pollack predictive coding (DKP-PC)<n>It simultaneously addresses both feedback delay and exponential decay, yielding a more efficient and scalable variant of PC.
  arXiv  Detail & Related papers  (2026-02-17T13:29:14Z)
• Towards Scaling Deep Neural Networks with Predictive Coding: Theory and Practice [1.2691047660244335]
  Backpropagation (BP) is the standard algorithm for training the deep neural networks that power modern artificial intelligence.<n>This thesis studies an alternative, potentially more efficient brain-inspired algorithm called predictive coding (PC)
  arXiv  Detail & Related papers  (2025-10-24T14:47:49Z)
• CollaPipe: Adaptive Segment-Optimized Pipeline Parallelism for Collaborative LLM Training in Heterogeneous Edge Networks [57.95170323315603]
  We introduce CollaPipe, a distributed learning framework that integrates collaborative pipeline parallelism with federated aggregation to support self-evolving networks.<n>In CollaPipe, the encoder part is adaptively partitioned into variable-sized segments and deployed across mobile devices for pipeline-parallel training, while the decoder is deployed on edge servers to handle generative tasks.<n>To enhance training efficiency, we formulate a joint optimization problem that adaptively allocates model segments, micro-batches, bandwidth, and transmission power.
  arXiv  Detail & Related papers  (2025-09-24T07:54:01Z)
• Bio-Inspired Artificial Neural Networks based on Predictive Coding [7.500043032028289]
  Backpropagation (BP) of errors is the backbone training algorithm for artificial neural networks (ANNs)<n>BP uses the chain rule to propagate the loss gradient backward through the network hierarchy, allowing efficient weight updates.<n> Predictive Coding (PC) has gained interest as a biologically plausible alternative that updates weights using only local information.
  arXiv  Detail & Related papers  (2025-08-12T09:09:16Z)
• ePC: Overcoming Exponential Signal Decay in Deep Predictive Coding Networks [9.400040788307223]
  Predictive Coding (PC) offers a biologically plausible alternative to backpropagation for neural network training.<n>This paper identifies the root cause and provides a principled solution.
  arXiv  Detail & Related papers  (2025-05-26T15:39:16Z)
• Tight Stability, Convergence, and Robustness Bounds for Predictive Coding Networks [60.3634789164648]
  Energy-based learning algorithms, such as predictive coding (PC), have garnered significant attention in the machine learning community. We rigorously analyze the stability, robustness, and convergence of PC through the lens of dynamical systems theory.
  arXiv  Detail & Related papers  (2024-10-07T02:57:26Z)
• Understanding Predictive Coding as an Adaptive Trust-Region Method [0.0]
  We develop a theory of PC as an adaptive trust-region (TR) algorithm that uses second-order information. We show that the learning dynamics of PC can be interpreted as interpolating between BP's loss gradient direction and a TR direction found by the PC inference dynamics.
  arXiv  Detail & Related papers  (2023-05-29T16:25:55Z)
• Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks [44.37047471448793]
  In this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL) We propose an innovative PSL framework, namely, efficient parallel split learning (EPSL) to accelerate model training. We show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy.
  arXiv  Detail & Related papers  (2023-03-26T16:09:48Z)
• Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks [50.68446003616802]
  Federated learning (FedL) has emerged as a popular technique for distributing model training over a set of wireless devices. We develop parallel successive learning (PSL), which expands the FedL architecture along three dimensions. Our analysis sheds light on the notion of cold vs. warmed up models, and model inertia in distributed machine learning.
  arXiv  Detail & Related papers  (2022-02-07T05:11:01Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• Dynamic Slimmable Network [105.74546828182834]
  We develop a dynamic network slimming regime named Dynamic Slimmable Network (DS-Net) Our DS-Net is empowered with the ability of dynamic inference by the proposed double-headed dynamic gate. It consistently outperforms its static counterparts as well as state-of-the-art static and dynamic model compression methods.
  arXiv  Detail & Related papers  (2021-03-24T15:25:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.