K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)

• URL: http://arxiv.org/abs/2607.05903v1
• Date: Tue, 07 Jul 2026 06:58:51 GMT
• Title: K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)
• Abstract summary: K-ABENAK-Adaptive Backpropagation with Error-based N-exclusion is a selective computation framework.<n>It reduces per-it computation training cost by excluding a fraction of low-loss ("minor") observations.<n>It collapses to 0.386 accuracy under label limitation (baseline: 0.832) and to 0.53 AUC under extreme imbalance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present K-ABENA (K-Adaptive Backpropagation with Error-based N-exclusion Algorithm), a selective gradient computation framework that reduces per-iteration training cost by excluding a fraction of low-loss ("minor") observations from the backward pass. Its canonical form (v3) combines a defensive-mixture sampling design over the minor set with Horvitz-Thompson inverse-probability reweighting, yielding a design-unbiased Horvitz-Thompson gradient estimator (Lemma 2) and whose self-normalized practical variant carries a bias of order O(1/m) with an explicit constant (Lemma 3). We prove an O(1/sqrt(T)) non-convex convergence guarantee for SGD under the estimator, with an additive term that quantifies the residual bias (Theorem 1). We further prove that uncompensated loss-based selection - a family that includes OHEM, SBP, and the two earlier K-ABENA variants - admits no stationary point at any minimizer where its selection bias is bounded away from zero (Proposition 2), and we quantify this failure empirically: at 0.17% class imbalance, uncompensated variants reach test AUC 0.53-0.62 versus 0.9998 for full-batch SGD, while the compensated estimator attains 0.9991 at identical 28.4% compute savings. On real datasets (Breast Cancer, Digits, Wine, Diabetes) the compensated estimator is statistically indistinguishable from full-batch SGD (paired permutation tests, p >= 0.5; Section 7) while saving 28-54% of per-epoch gradient computation. A biased "regularized mode" (the earlier half-domain variant) is retained as an option with a proven exact bias decomposition (Lemma 5) and quantified contraindications: it collapses to 0.386 accuracy under 40% label noise (baseline: 0.832) and to 0.53 AUC under extreme imbalance. Every advantage and every limitation reported in this paper is either proved or measured; all experiments are CPU-scale (NumPy/scikit-learn) and their scope is stated explicitly.

Related papers

• Semiparametric Inference for Conditional Shapley Feature Importance [0.0]
  This paper studies the conditional formulation, in which out-of-coalition features are integrated out under their true conditional distribution.<n>We propose a one-step estimator with K-fold cross-fitting and a U-statistic correction of the squared loss interval.<n>A Pinsker-type bound quantifies the bias from misspecifying the working copula class, while vine copulas keep conditional sampling tractable.
  arXiv  Detail & Related papers  (2026-09-09T15:19:24Z)
• Optimal Stratified Allocation for Rare-Event Onset Forecasting in Dependent Sequences [2.2632368327435732]
  We study estimation of total risk from a subsample K n under designs allocating K0 and K1 draws to the two strata.<n>We derive the exact finite-population variance of the weighted risk estimator under class-conditional sampling without replacement.
  arXiv  Detail & Related papers  (2026-09-03T19:38:01Z)
• EDGE: a closed-form directed test for the calibration of probabilistic binary classifiers [0.0]
  We propose EDGE, a calibration test for the canonical probabilistic classifier, logistic regression.<n> EDGE reads the same binned predicted-observed table a reliability diagram plots, and projects its standardized bin residuals onto a small pre-specified basis of smooth calibration-distortion.<n>Across link and feature misspecification the pre-specified default led or tied every rival binned test on the fitted index in 19 of 22 detectable scenarios, and stayed computable where the refit-based Stukel score test separates in 20% to 28% of samples.
  arXiv  Detail & Related papers  (2026-08-20T19:06:05Z)
• Hierarchical Empirical-Bayes Naive Bayes: Minimax Smoothing and Calibration with AODE Extension [2.423836984571605]
  We propose hierarchical empirical-Bayes Naive Bayes (HEB-NB)<n>Each class-feature conditional probability is smoothed by a Dirichlet prior whose concentration is learned data-adaptively.<n>Across 31 UCI and OpenML benchmarks, HEB-NB attains the best average Friedman rank on probabilistic metrics.
  arXiv  Detail & Related papers  (2026-08-11T17:21:31Z)
• Laplace-Fisher Gate Identities for Optimal Matrix-Gated Blended Score Estimation [0.0]
  Laplace-Fisher Gate Identity (LFGI)<n> finite-reference LFGI estimator to normalized density evaluation for Bayesian inverse problems.<n>Experiments using LFGI with known model evidence check absolute evidence calibration in both Gaussian and non-Gaussian settings.
  arXiv  Detail & Related papers  (2026-06-23T21:00:32Z)
• Bandit Convex Optimization with Gradient Prediction Adaptivity [56.816177049016794]
  We investigate whether optimistic gradient predictions can improve worst-case regret guarantees in a prediction-adaptive manner.<n>The key idea is a novel variance-reduced gradient estimator whose variance scales with the prediction error rather than the gradient norm.<n>We establish an information-theoretic lower bound that scales as $(sqrtmathbbE[S_T])$, providing a fundamental characterization of the best achievable prediction-adaptive regret.
  arXiv  Detail & Related papers  (2026-05-21T08:57:38Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Heuristic Pathologies and Further Variance Reduction via Uncertainty Propagation in the AIVAT Family of Techniques [62.94836578759359]
  We show how the uncertainty can be propagated to quantify the uncertainty of AIVAT estimates.<n>In our experiments, we use a dataset of 10,000 poker hands to demonstrate our pathology and uncertainty results.
  arXiv  Detail & Related papers  (2026-05-14T02:04:26Z)
• Bias-Corrected Adaptive Conformal Inference for Multi-Horizon Time Series Forecasting [0.0]
  Adaptive Conformal Inference (ACI) provides distribution-free prediction intervals with coverage guarantees for time series under distribution shift.<n>When a base forecaster develops persistent bias after a regime change, ACI compensates by widening intervals symmetrically, producing unnecessarily conservative bands.<n>We propose Bias-Corrected ACI (BC-ACI), which augments standard ACI with an exponentially weighted online moving average (EWM) estimate of forecast bias.
  arXiv  Detail & Related papers  (2026-04-14T19:33:59Z)
• Fixed-Horizon Self-Normalized Inference for Adaptive Experiments via Martingale AIPW/DML with Logged Propensities [0.0]
  Under adaptive assignment, propensities can keep changing, so the predictable quadratic variation of AIPW/DML increments may remain random.<n>We show that the Studentized statistic, with variance estimated by realized quadratic variation, is conditionally miscalibrated even without variance stabilization.
  arXiv  Detail & Related papers  (2026-02-17T13:12:31Z)
• Ensemble Threshold Calibration for Stable Sensitivity Control [0.0]
  We present an end-to-end framework that achieves exact recall with sub-percent variance over tens of millions of geometry pairs.<n>Our approach consistently hits a recall target within a small error, decreases redundant verifications relative to other calibrations, and runs end-to-end on a single TPU v3 core.
  arXiv  Detail & Related papers  (2025-10-02T15:22:28Z)
• Online Covariance Estimation in Nonsmooth Stochastic Approximation [14.818683408659764]
  We consider applying approximation (SA) methods to solve nonsmooth variational inclusion problems.<n>Our convergence construction establish the best-known for statistical estimation methods.
  arXiv  Detail & Related papers  (2025-02-07T20:16:51Z)
• Orthogonal Causal Calibration [55.28164682911196]
  We develop general algorithms for reducing the task of causal calibration to that of calibrating a standard (non-causal) predictive model.<n>Our results are exceedingly general, showing that essentially any existing calibration algorithm can be used in causal settings.
  arXiv  Detail & Related papers  (2024-06-04T03:35:25Z)
• Learning to Estimate Without Bias [57.82628598276623]
  Gauss theorem states that the weighted least squares estimator is a linear minimum variance unbiased estimation (MVUE) in linear models. In this paper, we take a first step towards extending this result to non linear settings via deep learning with bias constraints. A second motivation to BCE is in applications where multiple estimates of the same unknown are averaged for improved performance.
  arXiv  Detail & Related papers  (2021-10-24T10:23:51Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.