Fisher8: Stabilizing Neural Heteroscedastic Regression via Output-Layer Fisher Geometry

• URL: http://arxiv.org/abs/2608.10374v1
• Date: Tue, 11 Aug 2026 02:10:51 GMT
• Title: Fisher8: Stabilizing Neural Heteroscedastic Regression via Output-Layer Fisher Geometry
• Abstract summary: Training neural networks to jointly predict mean and uncertainty estimates can be unstable.<n>We derive Fisher8, an output-layer gradient correction that reorients and rescales updates using Fisher geometry rather than Euclidean geometry.<n>Across multidimensional regression and representation-learning tasks, Fisher8 obtains superior likelihood--error tradeoffs, predicts calibrated uncertainty estimates, and learns rich uncertainty-aware feature spaces.
• Score: 1.9336815376402718
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Training neural networks to jointly predict mean and uncertainty estimates from noisy observations can be unstable, prompting a series of independent stabilization efforts. We argue that these interventions highlight a common underlying issue where gradient steps are poorly aligned with the geometry of the loss landscape. To better align updates with local curvature, we derive Fisher8, an output-layer gradient correction that reorients and rescales updates using Fisher geometry rather than Euclidean geometry. Unlike past stabilizers, Fisher8 introduces no data-dependent hyperparameters beyond learning rate and admits an approximate KL trust radius between successive predictive distributions. We show that prior stabilizers converge on overlapping components of this geometric correction. Across multidimensional regression and representation-learning tasks, Fisher8 obtains superior likelihood--error tradeoffs, predicts calibrated uncertainty estimates, and learns rich uncertainty-aware feature spaces.

Related papers

• SPARC: Single-Pass Scaling for Motion Forecasting with Conformal Bayesian Last Layers [1.4452289368758382]
  Single-Pass Adaptive Risk is a structured, calibrated, and efficient uncertainty estimate for human motion forecasters.<n>A deterministic backbone predicts the future mean, and a conjugate Bayesian last layer converts time-domain feature leverage into an analytic horizon-wise scale.<n>It ranks first on NLL and on the combined MPJPE+NLL criterion while retaining competitive point accuracy and efficient calibrated tubes.
  arXiv  Detail & Related papers  (2026-08-21T07:20:22Z)
• Adaptive MPPI with Online Disturbance Covariance Estimation: Provable Stability Tightening via Spatial Smoothing [0.0]
  We study Model Predictive Path Integral (MPPI) control for nonlinear systems with additive process disturbances.<n>A mismatched disturbance covariance produces a persistent penalty in closed-loop stability certificates.
  arXiv  Detail & Related papers  (2026-07-09T21:12:40Z)
• RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration [14.194086645453767]
  Cross-pixel image registration is challenging due to severe non-linear radiometric and uncertainty.<n>In this paper we propose a novel framework that reformulates dense-modal flow estimation.
  arXiv  Detail & Related papers  (2026-06-29T15:58:25Z)
• Taming Curvature: Architecture Warm-Up for Stable Transformer Training [52.6838951211596]
  We introduce a fast online estimator of the largest (preconditioned) Hessian eigenvalue (i.e., curvature) based on a warm-started variant for power with Hessian-vector products.<n>We find that training instabilities coincide with surges in preconditioned curvature and that curvature grows with depth.<n> Experiments on large Transformers validate that our approach enables efficient curvature tracking and reduces instabilities.
  arXiv  Detail & Related papers  (2026-06-15T14:16:56Z)
• FEG-Pro: Forecast-Error Growth Profiling for Finite-Horizon Instability Analysis of Nonlinear Time Series [0.0]
  We propose FEG-Pro, a forecast-error growth profiling framework for nonlinear scalar time series.<n>The method constructs autocorrelation-guided sparse histories, performs distance-weighted k-nearest-neighbor multi-horizon forecasting, and analyzes the logarithmic growth of geometrically averaged forecast errors.<n>The pipeline also extracts the formal fit-selection regime, curvature, residual roughness after quadratic detrending, monotonicity, and forecast-error distribution entropy (FEDE) from signed multi-horizon errors.
  arXiv  Detail & Related papers  (2026-05-17T06:38:37Z)
• Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization [55.338784037660474]
  Multi- curvature reinforcement learning in domains requires balancing complex, non-fragile interference trade-offs between conflicting objectives.<n>In this paper we introduce a novel based real-time based real-time optimization on a robotic visual -- a proxy for monitoring ecosystems.
  arXiv  Detail & Related papers  (2026-05-12T21:32:57Z)
• FRInGe: Distribution-Space Integrated Gradients with Fisher--Rao Geometry [4.823870413661438]
  We propose Fisher--Rao Integrated Gradients (FRInGe), which defines both the reference and schedule in predictive distribution space.<n> FRInGe replaces input baselines with a maximum-entropy predictive reference and follows a Fisher-Rao geodesic on the probability simplex.
  arXiv  Detail & Related papers  (2026-05-07T15:12:15Z)
• DOC-GS: Dual-Domain Observation and Calibration for Reliable Sparse-View Gaussian Splatting [80.43237927269575]
  In this paper, we revisit sparse-view 3DGS reconstruction from a new perspective.<n>We identify the core challenge as the unobservability of Gaussian primitive reliability.<n>Motivated by this observation, we propose a unified Dual-domain Observation and geometrics in rendered images framework.
  arXiv  Detail & Related papers  (2026-04-08T07:01:24Z)
• Fisher-Geometric Diffusion in Stochastic Gradient Descent: Optimal Rates, Oracle Complexity, and Information-Theoretic Limits [0.0]
  We develop a theory of gradient descent in which mini-batch noise is an intrinsic, loss-induced matrix.<n>We prove oracle-complexity guarantees for epsilon-stationarity in the Fisher dual norm.
  arXiv  Detail & Related papers  (2026-03-02T21:57:09Z)
• VIKING: Deep variational inference with stochastic projections [48.946143517489496]
  Variational mean field approximations tend to struggle with contemporary overparametrized deep neural networks.<n>We propose a simple variational family that considers two independent linear subspaces of the parameter space.<n>This allows us to build a fully-correlated approximate posterior reflecting the overparametrization.
  arXiv  Detail & Related papers  (2025-10-27T15:38:35Z)
• Multivariate root-n-consistent smoothing parameter free matching estimators and estimators of inverse density weighted expectations [51.000851088730684]
  We develop novel modifications of nearest-neighbor and matching estimators which converge at the parametric $sqrt n $-rate.<n>We stress that our estimators do not involve nonparametric function estimators and in particular do not rely on sample-size dependent parameters smoothing.
  arXiv  Detail & Related papers  (2024-07-11T13:28:34Z)
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees [57.67528738886731]
  We study the numerical stability of scalable sparse approximations based on inducing points. For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
  arXiv  Detail & Related papers  (2022-10-14T15:20:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.