Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment

• URL: http://arxiv.org/abs/2605.30638v1
• Date: Thu, 28 May 2026 22:38:28 GMT
• Title: Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignment
• Abstract summary: We introduce a principled framework for broadcast-based credit assignment for general families of differentiable losses.<n>Error broadcast is a biologically plausible alternative to backpropagation that sends output information to hidden layers without weight transport.<n>We derive the cross-entropy case explicitly, characterize the admissible loss class, and introduce a score vector expansion technique that enriches the broadcast signal.
• Score: 34.9659273704306
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Score Broadcast and Decorrelation (SBD), a principled framework for broadcast-based credit assignment for general families of differentiable losses. Error broadcast is a biologically plausible alternative to backpropagation that sends output information to hidden layers without weight transport. The Error Broadcast and Decorrelation (EBD) framework, recently introduced for the mean-squared-error (MSE) setting, grounded this mechanism in the stochastic orthogonality of optimal estimators, under which the optimal residual is orthogonal to functions of the input. We generalize that foundation by introducing an orthogonality principle between the output score (the gradient of loss with respect to the final-layer output) and hidden-layer activations, which holds whenever the optimal score has conditional mean zero. This single principle unifies broadcast-based credit assignment across the standard differentiable-loss families, including cross-entropy, Bregman divergences, proper scoring rules, and exponential-family negative log-likelihoods. The framework supplies a theoretical grounding for the three-factor learning rule under general losses, with the neuromodulatory factor derived as the broadcast loss score. We derive the cross-entropy case explicitly, characterize the admissible loss class, and introduce a score vector expansion technique that enriches the broadcast signal while preserving the orthogonality framework. Experiments on CIFAR-10 and Tiny ImageNet show that SBD substantially improves over existing broadcast approaches, with score vector expansion delivering further gains. Overall, this work identifies the loss score as the signal to broadcast, supplies the orthogonality theory and theoretical grounding for the three-factor learning rule from neuroscience, and shows how score vector expansion enriches the decorrelation directions of the resulting objective.

Related papers

• CausalGate: Causal Importance Distillation for Transformer Module Pruning [0.0]
  CausalGate is an intervention-guided framework for compute-efficient runtime inference.<n>It measures semantic damage via the Kullback-Leibler divergence of the final logit distribution.<n>It consistently outperforms prominent dynamic routing and layer-skipping baselines.
  arXiv  Detail & Related papers  (2026-07-21T20:19:59Z)
• Rate-Distortion-Perception Theory: Redefining the Fundamental Limits of Information Representation [9.788112471288057]
  Rate-distortion-perception (RDP) theory extends the RD framework by introducing perception as a third fundamental axis.<n>Special attention is given to computational tools such as alternating schemes, Newton-based methods, and convex optimization.
  arXiv  Detail & Related papers  (2026-07-19T12:55:43Z)
• Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference [48.78219918881965]
  We propose a principled framework for deriving task-specific strictly proper scoring rules by matching the local curvature of the downstream error metric.<n>We apply this to the Average Treatment Effect (ATE) estimation, deriving a closed-form loss and its corresponding canonical probability mapping.
  arXiv  Detail & Related papers  (2026-06-02T08:41:25Z)
• DISA: Offline Importance Sampling for Distribution-Matching LLM-RL [56.9445657766829]
  We introduce DISA, which moves this calibration problem outside the RL loop.<n> DISA draws proposal trajectories offline, estimates the partition function via importance sampling, and freezes the resulting partition-function estimate.<n>On two open-weight backbones across six math and three code benchmarks, DISA matches or exceeds the online-coupled distribution-matching baseline Flow.
  arXiv  Detail & Related papers  (2026-05-17T07:14:44Z)
• Importance-Guided Basis Selection for Low-Rank Decomposition of Large Language Models [9.690793619550654]
  We present Basis Selection with Importance (BSI), a principled low-rank compression framework that ranks and prunes bases.<n>We develop an efficient Hessian-diagonal estimator by adapting the Hutchinson randomized-probing method to loss curvature with symmetric parameter perturbations.<n>We provide a comprehensive theoretical analysis, including loss-increase bounds under basis pruning, explicit propagation of Hessian-diagonal estimation error into these bounds, variance characterization tied to the Hessian spectrum, high-probability sample-complexity guarantees for achieving a target estimation accuracy, and guidance on perturbation intensity.
  arXiv  Detail & Related papers  (2026-05-02T22:35:02Z)
• Variational Rectification Inference for Learning with Noisy Labels [74.85528327499662]
  We propose variational rectification inference (VRI) to formulate the adaptive rectification for loss functions.<n>VRI is constructed as a hierarchical Bayes by treating the rectifying vector as a latent variable.<n>By introducing the variational term in VRI, the conditional posterior is estimated accurately and avoids collapsing to a Dirac delta function.
  arXiv  Detail & Related papers  (2026-03-18T01:25:08Z)
• The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss [53.542743390809356]
  This paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB)<n>Our analysis reveals a fundamental paradigm paradox: the more deterministic and structured the time series, the more severe the bias by point-wise loss function.<n>We present a concrete solution that simultaneously achieves both principles via DFT or DWT.
  arXiv  Detail & Related papers  (2025-12-21T06:08:22Z)
• Risk-Entropic Flow Matching [0.0]
  Tilted (entropic) risk, obtained by applying a log-exponential transform to a base loss, is a well established tool in statistics and machine learning.<n>We apply the standard risk-sensitive (log-exponential) transform to the conditional FM loss and show that the resulting tilted risk loss is a natural upper-bound on a meaningful conditional entropic FM objective.
  arXiv  Detail & Related papers  (2025-11-28T00:37:37Z)
• Error Broadcast and Decorrelation as a Potential Artificial and Natural Learning Mechanism [38.19776371710009]
  We introduce Error Broadcast and Decorrelation (EBD), a novel learning framework for neural networks.<n>EBD addresses credit assignment by directly broadcasting output errors to individual layers.<n>Our findings establish EBD as an efficient, biologically plausible, and principled alternative for neural network training.
  arXiv  Detail & Related papers  (2025-04-15T19:00:53Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.