Odds-Shift Slippage in One-vs-Rest Rankers: Diagnosing and Repairing Reweighting-Induced Top-K Errors

• URL: http://arxiv.org/abs/2609.13810v1
• Date: Sat, 12 Sep 2026 08:45:57 GMT
• Title: Odds-Shift Slippage in One-vs-Rest Rankers: Diagnosing and Repairing Reweighting-Induced Top-K Errors
• Abstract summary: One-vs-rest rankers that show each user the top-$K$ of many rare labels usually counter imbalance with a per-label positive-class weight.<n>We prove that a booster whose leaf steps are capped at $c$ realizes at most $Tc$ nat of shift in $T$ rounds at rate $$, which a cap sweep confirms.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One-vs-rest rankers that show each user the top-$K$ of many rare labels usually counter imbalance with a per-label positive-class weight, scale_pos_weight $= n_-/n_+$. Elkan's identity says such a weight shifts label $j$'s log-odds by $\ln w_j$, so the model ranks by weighted odds rather than by the marginal that is Bayes-optimal for precision@$K$, and suggests inverting the shift afterwards; what a finite learner does with a weight in the thousands, and which repair then works, has not been measured. We call the gap between the promised and the realized shift odds-shift slippage and measure it on matched pairs of LightGBM and MLP models that differ only in the weights. On Santander the weight takes MAP@7 from 0.808 to 0.117; for the boosted pairs the ideal odds shift accounts for 23% of that loss (32% on Instacart; 98% for an MLP pair on the same rows) and slippage for the rest. We prove that a booster whose leaf steps are capped at $c$ realizes at most $Tηc$ nat of shift in $T$ rounds at rate $η$, which a cap sweep confirms, and show that without a cap saturated cells tie at exactly 1.0, beyond the reach of any separable map. The analytic inversion therefore pays only where the shift was realized and nothing saturated, whereas per-label isotonic regression returns the Santander model to 0.784 (0.780 with the calibrator fitted on the validation period), but only if labels without calibration positives are mapped to their prior rather than passed through. On 11 public MULAN benchmarks and 5 learners the weighted model loses more than half of its MAP@$K$ in 8 of 55 cells, and on delicious and Corel5k the same repair returns it to the unweighted level; per-label calibration hurts where positives are scarce, a harm that a cross-validated rule removes. The recipe is released as oddslip.

Related papers

• Joint Affine Spectral Shaping: Coupling Weight and Bias Updates Beyond Weight-Only Muon [11.323131201168572]
  Matrix spectrals reshape weight-update spectra but usually delegate vector-valued biases to a separate.<n>We formulate each affine layer as a joint momentum matrix $A=[M_W,m_b]$ and apply a capped regularized-inverse spectral map to the complete matrix.<n>A strict five-seed ablation on a four-layer BERT-mini trained from scratch on IMDb compares exact-SVD Muon, weight-only inverse shaping, affine-probe inverse shaping, and the proposed joint regularized inverse.
  arXiv  Detail & Related papers  (2026-08-04T01:08:33Z)
• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• Covariance Last-Layer Ensembles: Function-Space Diversity for Efficient Uncertainty Quantification [0.8803840644459077]
  Cov-LLE restores function-space diversity that weight-orthonormality cannot, and at matched $K$ recovers much of the diversity and calibration of a deep ensemble.<n> Viewing OC as a last-layer ensemble also organizes detectors into a two-axis taxonomy.
  arXiv  Detail & Related papers  (2026-07-26T21:56:02Z)
• Hidden Boundary Motion in Transformer Optimization: Function-Space Orthogonalization of Affine Weight and Bias Updates [0.0]
  We call this hidden contribution emphboundary motion and decompose each update into a emphshape component and a shared emphboundary component.<n>In a single-seed experiment, SBO-AdamW raises validation accuracy from 81.68% to 85.81% and validation-selected test accuracy from 78.73% to 82.73%.
  arXiv  Detail & Related papers  (2026-07-24T21:51:43Z)
• Quantifying Diversity of Thought: A Predictive Law of Weighted LLM Ensemble Lift [0.0]
  This paper provides an experimentally verified formal law for calculating the uplift that diversity of thought provides in Large Language Model ensembles.<n>We test the law on 767,520 inferences from ten open-weight models over two graduate-level science benchmarks.
  arXiv  Detail & Related papers  (2026-07-19T19:01:06Z)
• Hidden-State Privacy Has an Empty Middle [51.56484100374058]
  We prove a complementary Fisher-ball lower bound: every full-rank Gaussian release at $O(1)$ Fisher utility admits a direction whose Mahalanobis signal grows linearly in hidden width.<n>A split-memory transformer trained from scratch reaches $G_mathrmMah in [20, 33]$ at 90M and maintains a $6$--$24times$ advantage over same-budget GPT baselines from 30M to 1B at a fixed-token language loss penalty.
  arXiv  Detail & Related papers  (2026-05-21T20:12:09Z)
• Bug or Feature$^2$: Weight Drift, Activation Sparsity and Spikes [53.726365933748134]
  We analyze a negative weight drift induced by the interaction between standard losses and positively biased activation functions.<n>We characterize the sparsity-accuracy tradeoff across 79 configurations and identify a sharp accuracy cliff above $sim$70% activation sparsity.
  arXiv  Detail & Related papers  (2026-05-17T21:29:20Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• When Does Margin Clamping Affect Training Variance? Dataset-Dependent Effects in Contrastive Forward-Forward Learning [0.0]
  Contrastive Forward-Forward (CFF) learning trains Vision Transformers layer by layer against supervised contrastive objectives.<n>We focus on one implementation detail: the positive-pair margin in the contrastive loss is applied through saturating similarity clamping.<n>We prove that an alternative formulation, subtracting the margin after the log-probability, is gradient-neutral under the mean-over-positives reduction.
  arXiv  Detail & Related papers  (2026-03-01T07:00:38Z)
• Optimal Unconstrained Self-Distillation in Ridge Regression: Strict Improvements, Precise Asymptotics, and One-Shot Tuning [61.07540493350384]
  Self-distillation (SD) is the process of retraining a student on a mixture of ground-truth and the teacher's own predictions.<n>We show that for any prediction risk, the optimally mixed student improves upon the ridge teacher for every regularization level.<n>We propose a consistent one-shot tuning method to estimate $star$ without grid search, sample splitting, or refitting.
  arXiv  Detail & Related papers  (2026-02-19T17:21:15Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlations [44.99833362998488]
  Changes in the data distribution at test time can have deleterious effects on the performance of predictive models. We propose a test-time label shift correction that adapts to changes in the joint distribution $p(y, z)$ using EM applied to unlabeled samples.
  arXiv  Detail & Related papers  (2022-11-28T18:52:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.