HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm

• URL: http://arxiv.org/abs/2605.24797v1
• Date: Sun, 24 May 2026 00:48:13 GMT
• Title: HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm
• Abstract summary: We propose a hierarchical and contrastive learning framework (HCL-FF) for deep neural networks.<n>HCL-FF introduces a coarse-to-fine hierarchical learning strategy that guides representations from low-level cues to high-level semantics.<n>Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that HCL-FF achieves new state-of-the-art performance among FF-based methods.
• Score: 23.026010149504387
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep neural networks trained with backpropagation have achieved outstanding performance in vision tasks but remain biologically implausible, computationally demanding, and difficult to interpret. The Forward-Forward (FF) algorithm offers a promising alternative by training each layer independently through local goodness objectives. However, its purely local optimization lacks hierarchical coordination across layers, and the decoupling of goodness from features leaves the representations unconstrained and semantically ambiguous. We propose a Hierarchical and Contrastive Learning FF framework (HCL-FF) to address these limitations. HCL-FF introduces (1) a coarse-to-fine hierarchical learning strategy that guides representations from low-level cues to high-level semantics, and (2) a supervised contrastive objective that enforces class-discriminative alignment after goodness decoupling. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that HCL-FF achieves new state-of-the-art performance among FF-based methods, with notable accuracy gains of +5.46%, +17.00%, and +12.51%, respectively.

Related papers

• Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignment [0.5249805590164902]
  We introduce a learnable channel-class assignment mechanism that enables adaptive, data-driven specialization of convolutional channels.<n>We also propose a loss-aware layer contribution strategy that adaptively weights intermediate-layer predictions based on their validation performance.
  arXiv  Detail & Related papers  (2026-06-07T14:24:07Z)
• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• Self-Distillation of Hidden Layers for Self-Supervised Representation Learning [18.042078225633045]
  We introduce Bootleg, a method that bridges the divide by tasking the model with predicting latent representations from hidden layers of a teacher network.<n>We demonstrate that Bootleg significantly outperforms comparable baselines on classification of ImageNet-1K and iNaturalist-21, and semantic segmentation of ADE20K and Cityscapes.
  arXiv  Detail & Related papers  (2026-03-16T17:13:27Z)
• Improving Visual Discriminability of CLIP for Training-Free Open-Vocabulary Semantic Segmentation [20.30263242388691]
  LHT-CLIP is a training-free framework that exploits the visual discriminability of CLIP across layer, head, and token levels.<n>It achieves state-of-the-art performance across diverse scenarios, highlighting its effectiveness and practicality for real-world deployment.
  arXiv  Detail & Related papers  (2025-10-27T22:05:08Z)
• Variational Supervised Contrastive Learning [50.79938854370321]
  We propose Variational Supervised Contrastive Learning (VarCon), which reformulates supervised contrastive learning as variational inference over latent class variables.<n>VarCon achieves state-of-the-art performance for contrastive learning frameworks, reaching 79.36% Top-1 accuracy on ImageNet-1K and 78.29% on CIFAR-100 with a ResNet-50 encoder.
  arXiv  Detail & Related papers  (2025-06-09T04:19:12Z)
• Self-Contrastive Forward-Forward Algorithm [3.1361717406527667]
  Forward-Forward (FF) algorithm relies on feedforward operations to optimize layer-wise objectives.<n>FF has failed to reach state-of-the-art performance on most standard benchmark tasks.<n>We propose Self-Contrastive Forward-Forward (SCFF) algorithm, a competitive training method aimed at closing this performance gap.
  arXiv  Detail & Related papers  (2024-09-17T22:58:20Z)
• Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm [5.1246638322893245]
  Forward-Forward (FF) algorithm has been proposed to alleviate the issues of backpropagation (BP) commonly used to train deep neural networks. We take the main ideas of FF and improve them by leveraging channel-wise competitive learning in the context of convolutional neural networks for image classification tasks. Our method outperforms recent FF-based models on image classification tasks, achieving testing errors of 0.58%, 7.69%, 21.89%, and 48.77% on MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100 respectively.
  arXiv  Detail & Related papers  (2023-12-19T23:48:43Z)
• Hierarchical Deep Counterfactual Regret Minimization [53.86223883060367]
  In this paper, we introduce the first hierarchical version of Deep CFR, an innovative method that boosts learning efficiency in tasks involving extensively large state spaces and deep game trees. A notable advantage of HDCFR over previous works is its ability to facilitate learning with predefined (human) expertise and foster the acquisition of skills that can be transferred to similar tasks.
  arXiv  Detail & Related papers  (2023-05-27T02:05:41Z)
• The Cascaded Forward Algorithm for Neural Network Training [61.06444586991505]
  We propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples. In our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems.
  arXiv  Detail & Related papers  (2023-03-17T02:01:11Z)
• Improving Music Performance Assessment with Contrastive Learning [78.8942067357231]
  This study investigates contrastive learning as a potential method to improve existing MPA systems. We introduce a weighted contrastive loss suitable for regression tasks applied to a convolutional neural network. Our results show that contrastive-based methods are able to match and exceed SoTA performance for MPA regression tasks.
  arXiv  Detail & Related papers  (2021-08-03T19:24:25Z)
• No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID Data [78.69828864672978]
  A central challenge in training classification models in the real-world federated system is learning with non-IID data. We propose a novel and simple algorithm called Virtual Representations (CCVR), which adjusts the classifier using virtual representations sampled from an approximated ssian mixture model. Experimental results demonstrate that CCVR state-of-the-art performance on popular federated learning benchmarks including CIFAR-10, CIFAR-100, and CINIC-10.
  arXiv  Detail & Related papers  (2021-06-09T12:02:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.