Introduction to Predictive Coding Networks for Machine Learning

• URL: http://arxiv.org/abs/2506.06332v1
• Date: Sat, 31 May 2025 04:48:53 GMT
• Title: Introduction to Predictive Coding Networks for Machine Learning
• Authors: Mikko Stenlund,
• Abstract summary: Predictive coding networks (PCNs) constitute a biologically inspired framework for understanding hierarchical computation in the brain.<n>We cover the foundational network architecture, inference and learning update rules, and algorithmic implementation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predictive coding networks (PCNs) constitute a biologically inspired framework for understanding hierarchical computation in the brain, and offer an alternative to traditional feedforward neural networks in ML. This note serves as a quick, onboarding introduction to PCNs for machine learning practitioners. We cover the foundational network architecture, inference and learning update rules, and algorithmic implementation. A concrete image-classification task (CIFAR-10) is provided as a benchmark-smashing application, together with an accompanying Python notebook containing the PyTorch implementation.

Related papers

• NNTile: a machine learning framework capable of training extremely large GPT language models on a single node [83.9328245724548]
  NNTile is based on a StarPU library, which implements task-based parallelism and schedules all provided tasks onto all available processing units.<n>It means that a particular operation, necessary to train a large neural network, can be performed on any of the CPU cores or GPU devices.
  arXiv  Detail & Related papers  (2025-04-17T16:22:32Z)
• Concept Learning in the Wild: Towards Algorithmic Understanding of Neural Networks [2.102973349909511]
  We study concept learning for an existing graph neural networks (GNN) model trained to solve Boolean satisfiability (SAT)<n>Our analysis reveals that the model learns key concepts matching those guiding human-designed SATs, particularly the notion of'support'<n>We use the discovered concepts to "reverse-engineer" the black-box GNN and rewrite it as a white-box textbook algorithm.
  arXiv  Detail & Related papers  (2024-12-15T14:37:56Z)
• Predictive Coding Networks and Inference Learning: Tutorial and Survey [0.7510165488300368]
  Predictive coding networks (PCNs) are based on the neuroscientific framework of predictive coding. Unlike traditional neural networks trained with backpropagation (BP), PCNs utilize inference learning (IL), a more biologically plausible algorithm. As inherently probabilistic (graphical) latent variable models, PCNs provide a versatile framework for both supervised learning and unsupervised (generative) modeling.
  arXiv  Detail & Related papers  (2024-07-04T18:39:20Z)
• Conditional computation in neural networks: principles and research trends [48.14569369912931]
  This article summarizes principles and ideas from the emerging area of applying textitconditional computation methods to the design of neural networks. In particular, we focus on neural networks that can dynamically activate or de-activate parts of their computational graph conditionally on their input.
  arXiv  Detail & Related papers  (2024-03-12T11:56:38Z)
• Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction [62.691996239590125]
  We propose an adaptive convolutional dictionary network (ACDNet) for metal artifact reduction. Our ACDNet can automatically learn the prior for artifact-free CT images via training data and adaptively adjust the representation kernels for each input CT image. Our method inherits the clear interpretability of model-based methods and maintains the powerful representation ability of learning-based methods.
  arXiv  Detail & Related papers  (2022-05-16T06:49:36Z)
• Continual Inference: A Library for Efficient Online Inference with Deep Neural Networks in PyTorch [97.03321382630975]
  Continual Inference is a Python library for implementing Continual Inference Networks (CINs) in PyTorch. We offer a comprehensive introduction to CINs and their implementation in practice, and provide best-practices and code examples for composing complex modules for modern Deep Learning.
  arXiv  Detail & Related papers  (2022-04-07T13:03:09Z)
• Predictive Coding: Towards a Future of Deep Learning beyond Backpropagation? [41.58529335439799]
  The backpropagation of error algorithm used to train deep neural networks has been fundamental to the successes of deep learning. Recent work has developed the idea into a general-purpose algorithm able to train neural networks using only local computations. We show the substantially greater flexibility of predictive coding networks against equivalent deep neural networks.
  arXiv  Detail & Related papers  (2022-02-18T22:57:03Z)
• FF-NSL: Feed-Forward Neural-Symbolic Learner [70.978007919101]
  This paper introduces a neural-symbolic learning framework, called Feed-Forward Neural-Symbolic Learner (FF-NSL) FF-NSL integrates state-of-the-art ILP systems based on the Answer Set semantics, with neural networks, in order to learn interpretable hypotheses from labelled unstructured data.
  arXiv  Detail & Related papers  (2021-06-24T15:38:34Z)
• Learning Structures for Deep Neural Networks [99.8331363309895]
  We propose to adopt the efficient coding principle, rooted in information theory and developed in computational neuroscience. We show that sparse coding can effectively maximize the entropy of the output signals. Our experiments on a public image classification dataset demonstrate that using the structure learned from scratch by our proposed algorithm, one can achieve a classification accuracy comparable to the best expert-designed structure.
  arXiv  Detail & Related papers  (2021-05-27T12:27:24Z)
• Introduction to Machine Learning for the Sciences [0.0]
  The notes start with an exposition of machine learning methods without neural networks, such as principle component analysis, t-SNE, and linear regression. We continue with an introduction to both basic and advanced neural network structures such as conventional neural networks, (variational) autoencoders, generative adversarial networks, restricted Boltzmann machines, and recurrent neural networks.
  arXiv  Detail & Related papers  (2021-02-08T16:25:46Z)
• Local Critic Training for Model-Parallel Learning of Deep Neural Networks [94.69202357137452]
  We propose a novel model-parallel learning method, called local critic training. We show that the proposed approach successfully decouples the update process of the layer groups for both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) We also show that trained networks by the proposed method can be used for structural optimization.
  arXiv  Detail & Related papers  (2021-02-03T09:30:45Z)
• Applications of Deep Neural Networks with Keras [0.0]
  Deep learning allows a neural network to learn hierarchies of information in a way that is like the function of the human brain. This course will introduce the student to classic neural network structures, Conversa Neural Networks (CNN), Long Short-Term Memory (LSTM), Gated Recurrent Neural Networks (GRU), General Adrial Networks (GAN)
  arXiv  Detail & Related papers  (2020-09-11T22:09:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.