FFR: Forward-Forward Learning for Regression

• URL: http://arxiv.org/abs/2606.03927v1
• Date: Tue, 02 Jun 2026 17:15:59 GMT
• Title: FFR: Forward-Forward Learning for Regression
• Abstract summary: We propose FFR (Forward-Forward for Regression), to our knowledge, the first framework to extend FF to real-world regression.<n>FFR introduces three key innovations: (1) an ordinal competitive goodness function that replaces contrastive pairs with competitive learning between partitioned neuron groups under distance-aware supervision; and (2) a stratified ladder architecture where shallow layers learn coarse ordinal discrimination and deeper layers refine into fine-grained regression.
• Score: 24.116997757752998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The Forward-Forward (FF) algorithm offers a computationally efficient and biologically plausible alternative to backpropagation (BP) by training neural networks through purely local, layer-wise optimization. However, FF is inherently designed for classification via contrastive positive-negative sample pairs, and extending it to regression poses fundamental challenges: continuous target space lack natural "opposites" for contrastive learning, and the standard goodness function carries no information about target magnitude or ordering. We propose FFR (Forward-Forward for Regression), to our knowledge, the first framework to extend FF to real-world regression and demonstrate competitive performance across diverse real-world datasets. FFR introduces three key innovations: (1) an ordinal competitive goodness function that replaces contrastive pairs with competitive learning between partitioned neuron groups under distance-aware ordinal supervision; (2) a stratified ladder architecture where shallow layers learn coarse ordinal discrimination and deeper layers refine into fine-grained regression, with multi-scale feature aggregation for inter-layer collaboration; and (3) hierarchical prediction with uncertainty estimation, where multi-scale predictors jointly provide robust predictions and prediction confidence as a free-lunch. Extensive experimental results show FFR recovers on average 98.6% of BP's accuracy across five real-world regression benchmarks while reducing peak training memory to only 27% of BP's at depth 8 and 8% at depth 32, with per-iteration time around 72% of BP's, and substantially outperforms all BP-free competitors.

Related papers

• Customer Churn Prediction on Structured Data Using FT-Transformer and Stacking Ensembles [0.0]
  This study introduces a validated hybrid architecture that integrates feature-tokenized transformers (FT-Transformer) with gradient-boosted trees through calibration stacking.<n>The proposed framework addresses persistent gaps in statistical validation, probability, calibration, and found in prior research.<n>On a public bank churn dataset, the hybrid model 62.10% F1, 0.861 AUC-ROC, and 0.647 PR-AUC, outperforming the Multi-Layer Perceptron (MLP) baseline by 3.37 F1 points and 0.027 AUC with 95% confidence intervals reported.
  arXiv  Detail & Related papers  (2026-05-26T23:02:38Z)
• FedSQ: Optimized Weight Averaging via Fixed Gating [1.2058208023553034]
  Federated learning enables collaborative training across organizations without sharing raw data.<n>We propose FedSQ (Federated Structural-Quantitative learning), a transfer-d neural federated procedure based on a DualCopy.<n>Fixing the gating reduces learning to within-regime affine refinements, which stabilizes aggregation under heterogeneous partitions.
  arXiv  Detail & Related papers  (2026-04-03T11:54:23Z)
• REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge [83.2858110368572]
  We propose textbfREAL (underlineREgression-underlineAware Reinforcement underlineLThought), a principled RL framework designed to optimize regression rewards.<n>We show that REAL consistently outperforms both regression-aware SFT baselines and standard RL methods.
  arXiv  Detail & Related papers  (2026-03-17T21:19:08Z)
• DeFRiS: Silo-Cooperative IoT Applications Scheduling via Decentralized Federated Reinforcement Learning [62.347535250646196]
  This paper proposes DeFRiS, a Decentralized Federated Reinforcement Learning framework for robust and scalable Silo-cooperative IoT application scheduling.<n>DeFRiS integrates three innovations: (i) an action-space-agnostic policy utilizing candidate resource scoring to enable seamless knowledge transfer across heterogeneous silos; (ii) a silo-optimized local learning mechanism combining Generalized Advantage Estimation with clipped policy updates to resolve sparse delayed reward challenges; and (iii) a Dual-Track Non-IID robust decentralized aggregation protocol leveraging gradient fingerprints for similarity-aware knowledge transfer and anomaly detection.
  arXiv  Detail & Related papers  (2026-03-16T02:02:38Z)
• Enhancing Continual Learning for Software Vulnerability Prediction: Addressing Catastrophic Forgetting via Hybrid-Confidence-Aware Selective Replay for Temporal LLM Fine-Tuning [43.24339861841546]
  This paper investigates fine-tuning of a decoder-style language model (phi/phi-2 with LoRA) on a CVE-linked dataset.<n>We evaluate eight continual learning strategies, including window-only and cumulative training, replay-based baselines and regularisation-based variants.<n>Hybrid-CASR reduces training time per window by about 17 percent compared to the baseline, whereas cumulative training delivers only a minor F1 increase (0.661) at a 15.9-fold computational cost.
  arXiv  Detail & Related papers  (2026-02-27T09:13:23Z)
• Energy-Efficient Deep Learning Without Backpropagation: A Rigorous Evaluation of Forward-Only Algorithms [0.0]
  We present evidence that the Mono-Forward algorithm consistently surpasses optimally tuned BP baseline in classification accuracy.<n>This superior generalization is achieved with profound efficiency gains, including up to 41% less energy consumption and up to 34% faster training.
  arXiv  Detail & Related papers  (2025-11-02T19:48:44Z)
• Beyond Backpropagation: Exploring Innovative Algorithms for Energy-Efficient Deep Neural Network Training [0.0]
  The paper rigorously investigates three BP-free training methods: the Forward-Forward (FF), Cascaded-Forward (CaFo), and Mono-Forward (MF)<n>MF reduces energy consumption by up to 41% and shortens training time by up to 34%, translating to a measurably smaller carbon footprint as estimated by CodeCarbon.
  arXiv  Detail & Related papers  (2025-09-23T14:27:44Z)
• Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation [5.092009068303438]
  We propose a new Forward-Forward (FF)-based training framework tailored for convolutional neural networks (CNNs)<n>ASGE features maps to compute spatially-aware goodness rep- resentations at each layer, enabling layer-wise supervision.<n>We present the first successful ap- plication of FF-based training to ImageNet datasets, with Top-1 and Top-5 accuracies of 26.21% and 47.49%.
  arXiv  Detail & Related papers  (2025-09-15T19:38:32Z)
• Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation [70.43845294145714]
  Relieving the reliance of neural network training on a global back-propagation (BP) has emerged as a notable research topic. We propose a local training strategy that successively regularizes the gradient reconciliation between neighboring modules. Our method can be integrated into both local-BP and BP-free settings.
  arXiv  Detail & Related papers  (2024-06-07T19:10:31Z)
• The Cascaded Forward Algorithm for Neural Network Training [61.06444586991505]
  We propose a new learning framework for neural networks, namely Cascaded Forward (CaFo) algorithm, which does not rely on BP optimization as that in FF. Unlike FF, our framework directly outputs label distributions at each cascaded block, which does not require generation of additional negative samples. In our framework each block can be trained independently, so it can be easily deployed into parallel acceleration systems.
  arXiv  Detail & Related papers  (2023-03-17T02:01:11Z)
• Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach [66.9033666087719]
  This paper extends the inference view and describes a variational inference formulation of federated learning. We apply FedEP on standard federated learning benchmarks and find that it outperforms strong baselines in terms of both convergence speed and accuracy.
  arXiv  Detail & Related papers  (2023-02-08T17:58:11Z)
• Heterogeneous Federated Learning via Grouped Sequential-to-Parallel Training [60.892342868936865]
  Federated learning (FL) is a rapidly growing privacy-preserving collaborative machine learning paradigm. We propose a data heterogeneous-robust FL approach, FedGSP, to address this challenge. We show that FedGSP improves the accuracy by 3.7% on average compared with seven state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-01-31T03:15:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.