Hyperspherical Forward-Forward with Prototypical Representations

• URL: http://arxiv.org/abs/2605.00082v1
• Date: Thu, 30 Apr 2026 15:33:37 GMT
• Title: Hyperspherical Forward-Forward with Prototypical Representations
• Abstract summary: We introduce the Hyperspherical Forward-Forward (HFF), a novel reformulation that resolves this critical bottleneck.<n>Our core innovation is to reframe the local objective from a binary goodness-of-fit task to a direct multi-class classification problem.<n>We are among the first greedy local-learning methods to report over 25% top-1 accuracy on ImageNet-1k, and 65.96% with transfer learning.
• Score: 7.053013122947757
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Forward-Forward (FF) algorithm presents a compelling, bio-inspired alternative to backpropagation. However, while efficient in training, it has a computationally prohibitive inference process that requires a separate forward pass for every class that is evaluated. In this work, we introduce the Hyperspherical Forward-Forward (HFF), a novel reformulation that resolves this critical bottleneck. Our core innovation is to reframe the local objective of each layer from a binary goodness-of-fit task to a direct multi-class classification problem within a hyperspherical feature space. We achieve this by learning a set of class-specific, unit-norm prototypes that act as geometric anchors and implicit negatives. This architectural innovation preserves the benefits of local training while enabling weight update and inference in a single forward pass, making it >40x faster than the original FF algorithm. Our method is simple to implement, scales effectively to modern convolutional architectures, and achieves superior accuracy on standard image classification benchmarks, closing the gap with backpropagation. Most notably, we are among the first greedy local-learning methods to report over 25% top-1 accuracy on ImageNet-1k, and 65.96% with transfer learning.

Related papers

• Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization [49.06421851486415]
  Static sparse training aims to train sparse models from scratch, achieving remarkable results in recent years. We propose Exact Orthogonal Initialization (EOI), a novel sparse Orthogonal Initialization scheme based on random Givens rotations. Our method enables training highly sparse 1000-layer and CNN networks without residual connections or normalization techniques.
  arXiv  Detail & Related papers  (2024-06-03T19:44:47Z)
• Class-Imbalanced Semi-Supervised Learning for Large-Scale Point Cloud Semantic Segmentation via Decoupling Optimization [64.36097398869774]
  Semi-supervised learning (SSL) has been an active research topic for large-scale 3D scene understanding. The existing SSL-based methods suffer from severe training bias due to class imbalance and long-tail distributions of the point cloud data. We introduce a new decoupling optimization framework, which disentangles feature representation learning and classifier in an alternative optimization manner to shift the bias decision boundary effectively.
  arXiv  Detail & Related papers  (2024-01-13T04:16:40Z)
• FDCNet: Feature Drift Compensation Network for Class-Incremental Weakly Supervised Object Localization [10.08410402383604]
  This work addresses the task of class-incremental weakly supervised object localization (CI-WSOL) The goal is to incrementally learn object localization for novel classes using only image-level annotations while retaining the ability to localize previously learned classes. We first present a strong baseline method for CI-WSOL by adapting the strategies of class-incremental classifiers to catastrophic forgetting. We then propose the feature drift compensation network to compensate for the effects of feature drifts on class scores and localization maps.
  arXiv  Detail & Related papers  (2023-09-17T01:10:45Z)
• The Cascaded Forward Algorithm for Neural Network Training [61.06444586991505]
  We propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples. In our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems.
  arXiv  Detail & Related papers  (2023-03-17T02:01:11Z)
• Towards Calibrated Hyper-Sphere Representation via Distribution Overlap Coefficient for Long-tailed Learning [8.208237033120492]
  Long-tailed learning aims to tackle the challenge that head classes dominate the training procedure under severe class imbalance in real-world scenarios. Motivated by this, we generalize the cosine-based classifiers to a von Mises-Fisher (vMF) mixture model. We measure representation quality upon the hyper-sphere space via calculating distribution overlap coefficient.
  arXiv  Detail & Related papers  (2022-08-22T03:53:29Z)
• Efficient Few-Shot Object Detection via Knowledge Inheritance [62.36414544915032]
  Few-shot object detection (FSOD) aims at learning a generic detector that can adapt to unseen tasks with scarce training samples. We present an efficient pretrain-transfer framework (PTF) baseline with no computational increment. We also propose an adaptive length re-scaling (ALR) strategy to alleviate the vector length inconsistency between the predicted novel weights and the pretrained base weights.
  arXiv  Detail & Related papers  (2022-03-23T06:24:31Z)
• Fast Few-Shot Classification by Few-Iteration Meta-Learning [173.32497326674775]
  We introduce a fast optimization-based meta-learning method for few-shot classification. Our strategy enables important aspects of the base learner objective to be learned during meta-training. We perform a comprehensive experimental analysis, demonstrating the speed and effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-10-01T15:59:31Z)
• Prior Guided Feature Enrichment Network for Few-Shot Segmentation [64.91560451900125]
  State-of-the-art semantic segmentation methods require sufficient labeled data to achieve good results. Few-shot segmentation is proposed to tackle this problem by learning a model that quickly adapts to new classes with a few labeled support samples. Theses frameworks still face the challenge of generalization ability reduction on unseen classes due to inappropriate use of high-level semantic information.
  arXiv  Detail & Related papers  (2020-08-04T10:41:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.