K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networks

• URL: http://arxiv.org/abs/2604.11011v1
• Date: Mon, 13 Apr 2026 05:24:44 GMT
• Title: K-Way Energy Probes for Metacognition Reduce to Softmax in Discriminative Predictive Coding Networks
• Abstract summary: We present an approximate reduction showing that with target-clamped CE-energy training and effectively-feedforward latent dynamics, the K-way energy margin decomposes into a monotone function of the log-max margin margin plus a residual that is not trained to correlate with correctness.<n>We test this across six conditions on CIFAR-10: extended deterministic training, direct measurement of latent movement during inference, a post-hoc decoder fairness control on a backpropagation network, a matched-budget PC vs BP comparison, a five-point Langevin temperature sweep, and trajectory-integrated MCPC training.<n>The gap was stable
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present this as a negative result with an explanatory mechanism, not as a formal upper bound. Predictive coding networks (PCNs) admit a K-way energy probe in which each candidate class is fixed as a target, inference is run to settling, and the per-hypothesis settled energies are compared. The probe appears to read a richer signal source than softmax, since the per-hypothesis energy depends on the entire generative chain. We argue this appearance is misleading under the standard Pinchetti-style discriminative PC formulation. We present an approximate reduction showing that with target-clamped CE-energy training and effectively-feedforward latent dynamics, the K-way energy margin decomposes into a monotone function of the log-softmax margin plus a residual that is not trained to correlate with correctness. The decomposition predicts that the structural probe should track softmax from below. We test this across six conditions on CIFAR-10: extended deterministic training, direct measurement of latent movement during inference, a post-hoc decoder fairness control on a backpropagation network, a matched-budget PC vs BP comparison, a five-point Langevin temperature sweep, and trajectory-integrated MCPC training. In every condition the probe sat below softmax. The gap was stable across training procedures within the discriminative PC family. Final-state and trajectory-integrated training produced probes whose AUROC_2 values differed by less than 10^-3 at deterministic evaluation. The empirical regime is small: single seed, 2.1M-parameter network, 1280 test images. We frame the result as a preprint inviting replication. We discuss conditions under which the decomposition does not apply (bidirectional PC, prospective configuration, generative PC, non-CE energy formulations) and directions for productive structural probing the analysis does not foreclose.

Related papers

• Trajectory-Anchor Optimization for Overconfident Thermal Visual Place Recognition: Zero-Leakage OOD Auditing and Kidnapped-Robot Recovery [3.9929096403817255]
  Trajectory-Anchor Optimization (TAO) is an efficient fail-safe filter at a macro-scale.<n>Within a 5m radius, hallucinations often possess a locally consistent geometry that deceives rigid alignment.<n>TAO establishes a distinct convergence basin (10m) where multi-view consistency reliably isolates catastrophic topological breaks.
  arXiv  Detail & Related papers  (2026-07-06T07:31:37Z)
• Mind the Residual Gap: Probabilistic Downscaling under Real-World Bias [11.400279490725042]
  Probabilistic downscaling is a central challenge to atmospheric science, climate modeling, and other multiscale physical systems.<n>A widely used paradigm decomposes the problem into a deterministic mean predictor followed by a residual generator.<n>We show that the root cause is residual target miscalibration, the residual distribution induced during training differs systematically from the one required at test time due to downscaling bias.
  arXiv  Detail & Related papers  (2026-06-29T18:47:34Z)
• Direct Bethe Free Energy Minimization for Bayesian Neural Network [0.0]
  We propose training Bayesian neural networks by directly minimizing the Bethe free energy rather than maximizing a variational lower bound.<n>On tree-structured factor graphs the Bethe free energy is exact; deterministic layers drop out of the objective and are trained by standard backpropagation.<n>All variants admit a closed-form predictive at MAP-equivalent inference cost, in contrast to ensemble and sampling-based methods.
  arXiv  Detail & Related papers  (2026-05-08T20:12:48Z)
• Cross-Entropy Is Load-Bearing: A Pre-Registered Scope Test of the K-Way Energy Probe on Bidirectional Predictive Coding [0.0]
  Cacioli, 2026, showed that the K-way energy probe on standard discriminative predictive coding networks reduces approximately to a monotone function of the log-softmax margin.<n>This study tests the reduction's sensitivity to CE removal using two conditions: standard PC trained with MSE instead of CE, and bidirectional PC.
  arXiv  Detail & Related papers  (2026-04-23T05:03:18Z)
• Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration [76.08899010904652]
  CapCal is a training-free framework that mechanically decouples positional bias from ranking decisions.<n>It achieves superior performance among training-free methods while preserving single-pass efficiency.
  arXiv  Detail & Related papers  (2026-04-11T10:47:22Z)
• Softmax is not Enough (for Adaptive Conformal Classification) [5.184894950445513]
  Conformal Prediction (CP) is a distribution-free framework for uncertainty quantification.<n>Nonconformity scores are derived from softmax outputs, which can be unreliable indicators of how certain the model truly is about a given input.<n>We propose a new approach that leverages information from the pre-softmax logit space, using the Helmholtz Free Energy as a measure of model uncertainty and sample difficulty.
  arXiv  Detail & Related papers  (2026-02-23T04:33:04Z)
• Bayesian Uncertainty Quantification with Anchored Ensembles for Robust EV Power Consumption Prediction [0.8602553195689513]
  EV power estimation underpins range prediction and energy management, yet practitioners need both point accuracy and trustworthy uncertainty.<n>We propose an anchored-ensemble Long Short-Term Memory (LSTM) with a Student-t likelihood that jointly captures uncertainty.<n>Our model attains strong accuracy: RMSE 3.36 +/- 1.10, MAE 2.21 +/- 0.89, R-squared = 0.93 +/- 0.02, explained variance 0.93 +/- 0.02, and delivers well-calibrated uncertainty bands with near-nominal coverage.
  arXiv  Detail & Related papers  (2025-11-09T20:49:42Z)
• Assessing the Limits of Graph Neural Networks for Vapor-Liquid Equilibrium Prediction: A Cryogenic Mixture Case Study [3.765010805872486]
  This study asks whether a structure-aware graph neural network (GNN; DimeNet++) trained on GERG-2008/CoolProp data can act as a practical surrogate for an equation of state (EoS)<n>We generate a ternary dataset over 90-200 K and pressures to 100 bar, curate it with a 15% density filter (reducing 5,200 states to 1,516), and pair each state with a lightweight molecular-dynamics snapshot to supply structural features.<n>We conclude that, as configured, the surrogate in this study is not equilibrium-ready for VLE and offers no runtime benefit.
  arXiv  Detail & Related papers  (2025-09-10T16:10:58Z)
• A New Formulation of Lipschitz Constrained With Functional Gradient Learning for GANs [52.55025869932486]
  This paper introduces a promising alternative method for training Generative Adversarial Networks (GANs) on large-scale datasets with clear theoretical guarantees.<n>We propose a novel Lipschitz-constrained Functional Gradient GANs learning (Li-CFG) method to stabilize the training of GAN.<n>We demonstrate that the neighborhood size of the latent vector can be reduced by increasing the norm of the discriminator gradient.
  arXiv  Detail & Related papers  (2025-01-20T02:48:07Z)
• Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shifts [8.431465371266391]
  Density-Softmax is a sampling-free deterministic framework for uncertainty estimation. We show that our model is the solution of minimax uncertainty risk. Our method enjoys competitive results with state-of-the-art techniques in terms of uncertainty and robustness.
  arXiv  Detail & Related papers  (2023-02-13T16:21:03Z)
• Combating Mode Collapse in GANs via Manifold Entropy Estimation [70.06639443446545]
  Generative Adversarial Networks (GANs) have shown compelling results in various tasks and applications. We propose a novel training pipeline to address the mode collapse issue of GANs.
  arXiv  Detail & Related papers  (2022-08-25T12:33:31Z)
• Distinction Maximization Loss: Efficiently Improving Classification Accuracy, Uncertainty Estimation, and Out-of-Distribution Detection Simply Replacing the Loss and Calibrating [2.262407399039118]
  We propose training deterministic deep neural networks using our DisMax loss. DisMax usually outperforms all current approaches simultaneously in classification accuracy, uncertainty estimation, inference efficiency, and out-of-distribution detection.
  arXiv  Detail & Related papers  (2022-05-12T04:37:35Z)
• On the Practicality of Differential Privacy in Federated Learning by Tuning Iteration Times [51.61278695776151]
  Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively. Recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks.
  arXiv  Detail & Related papers  (2021-01-11T19:43:12Z)
• Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias [65.13042449121411]
  In practice, training a network with the gradient estimates provided by EP does not scale to visual tasks harder than MNIST. We show that a bias in the gradient estimate of EP, inherent in the use of finite nudging, is responsible for this phenomenon. We apply these techniques to train an architecture with asymmetric forward and backward connections, yielding a 13.2% test error.
  arXiv  Detail & Related papers  (2020-06-06T09:36:07Z)
• TRP: Trained Rank Pruning for Efficient Deep Neural Networks [69.06699632822514]
  We propose Trained Rank Pruning (TRP), which alternates between low rank approximation and training. A nuclear regularization optimized by sub-gradient descent is utilized to further promote low rank in TRP. The TRP trained network inherently has a low-rank structure, and is approximated with negligible performance loss.
  arXiv  Detail & Related papers  (2020-04-30T03:37:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.