Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation

• URL: http://arxiv.org/abs/2606.29471v1
• Date: Sun, 28 Jun 2026 15:57:59 GMT
• Title: Structured Proper Loss Geometries for Multiclass Classification: Theory and Controlled Empirical Evaluation
• Abstract summary: We study three multiclass objectives: a class-aware quadratic Bregman score (CAPM), a strongly convex generator with constrained log-cosh ridges (HPG), and an HPG objective with a probability-margin penalty (APMS)<n>We derive conditional-regret, curvature, range, and logit-gradient bounds for CAPM and HPG, and prove exact penalty-range and conditional-target displacement bounds for APMS.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Strictly proper scoring rules identify the true conditional class distribution at population level, but their curvature can alter optimization and finite-sample behavior. We study three multiclass objectives: a class-aware quadratic Bregman score (CAPM), a strongly convex generator with constrained log-cosh ridges (HPG), and an HPG objective with an annealed probability-margin penalty (APMS). CAPM is treated as a structured instance of established quadratic scoring-rule theory. We derive conditional-regret, curvature, range, and logit-gradient bounds for CAPM and HPG, and prove exact penalty-range and conditional-target displacement bounds for APMS. Controlled five-seed experiments use Digits, Wisconsin breast cancer, and synthetic confusion and long-tail problems under clean labels, symmetric and pair-flip corruption, class imbalance, calibration evaluation, input corruption, and first-order adversarial perturbations. The candidates are close to cross-entropy on clean data and show descriptive gains in some noisy-label cells, but the five-seed comparisons are interpreted descriptively rather than as significance evidence. The selected noisy-label baselines perform better on Digits with 40% symmetric label noise, and explicit prior-adjustment methods perform better in the 30:1 synthetic long-tail experiment. Ablations do not show a consistent benefit from the candidate-specific graph, ridge, or margin components. The mathematical analysis establishes the stated properties, and the experiments delimit the empirical evidence; together they do not support a claim of general superiority.

Related papers

• COMPACT: Spectral Adjustment Scores from a Complete and Irreducible Causal Criterion [1.8275108630751837]
  Propensity scores can simplify adjustment by reducing high-dimensional covariates to a scalar with binary treatment.<n>Under faithfulness, we identify the largest set of unconditional and conditional dependence relations whose truth is invariant to whether treatment causes the outcome, leaving treatment-effect estimation to the downstream analysis.<n>We show that sufficiently informative proxies can recover this span without direct observation of the adjustment variables, characterize the resulting estimation and causal errors, and establish validity for the complete procedure.
  arXiv  Detail & Related papers  (2026-08-10T23:22:20Z)
• A Fuzzy Rule-based Neuro-Symbolic Approach for Pipe Severity Prediction in Sewer Networks [0.0]
  This study introduces a modular, fuzzy rule-based neuro-symbolic framework that bridges this gap by decoupling neural perception from symbolic reasoning.<n>For reasoning, a DT, specifically Weka's J48, algorithm is trained on ground-truth CODEs and severity labels, and its paths are converted into 19 fixed IF--THEN rules.<n>Overall, the framework combines competitive class-balanced performance with traceable reasoning from predicted CODE degrees to rule supports and severity evidence.
  arXiv  Detail & Related papers  (2026-07-30T16:33:26Z)
• RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification [38.8278639079174]
  Class imbalance poses a fundamental challenge in risk-sensitive applications such as fraud detection and medical diagnosis.<n>We introduce RUBRIC, a generator-agnostic filtering framework that formulates synthetic sample selection as a quality-over-quantity optimization problem.<n>We show that RUBRIC improves F1-macro and recall while maintaining comparable ROC-AUC across several generators.
  arXiv  Detail & Related papers  (2026-07-10T06:37:12Z)
• The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning [0.5854803320592717]
  We prove optimality in a linear-Gaussian model (Thm. A), necessity of range coverage for any quadratic penalty that zeros deployment drift (Thm. G)<n>Thirteen blocks (ML through Qwen2.5-7B) test matched vs isotropic vs wrong-direction penalties on geometry and deployment drift.<n>We claim a falsifiable design recipe: estimate Sigma_task, match Sigma', run the controls, report task and geometry separately.
  arXiv  Detail & Related papers  (2026-05-21T17:53:28Z)
• Bootstrapping with AI/ML-generated labels [0.0]
  We show that a seemingly natural fixed-label bootstrap, which generates data using estimated labels but relies on a corrupted version in estimation, is generally invalid unless a strong independence condition holds.<n>We propose a coupled-label bootstrap that jointly resamples the true and imputed labels, and show it is valid without this condition.<n>We illustrate the methods in simulations and apply them to investigate the relationship between wages and remote work status.
  arXiv  Detail & Related papers  (2026-04-26T15:48:03Z)
• LiveMathematicianBench: A Live Benchmark for Mathematician-Level Reasoning with Proof Sketches [61.30693283718321]
  We present LiveMathematicianBench, a dynamic multiple-choice benchmark for research-level mathematical reasoning.<n>By grounding evaluation in newly published theorems, it provides a realistic testbed beyond memorized patterns.<n>It employs a proof-sketchguided distractor pipeline that uses high-level proof strategies to construct plausible but invalid answer choices.
  arXiv  Detail & Related papers  (2026-04-02T08:22:17Z)
• Differential Voting: Loss Functions For Axiomatically Diverse Aggregation of Heterogeneous Preferences [6.3240435869587515]
  Reinforcement learning from human feedback can be viewed as a form of voting, where the aggregation mechanism is defined by the loss function.<n>We introduce Differential Voting, a framework that constructs instance-wise, differentiable loss functions whose population-level optima provably correspond to classical voting rules.<n>Our analysis shows how design choices in loss geometry-such as margin sensitivity and boundary concentration-directly translate into normative aggregation behavior.
  arXiv  Detail & Related papers  (2026-01-25T03:59:51Z)
• Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness [61.45587642780908]
  We propose a three-step approach for parameter-efficient fine-tuning of image-text foundation models.<n>Our method improves its two key components: minority samples identification and the robust training algorithm.<n>Our theoretical analysis shows that our PPA enhances minority group identification and is Bayes optimal for minimizing the balanced group error.
  arXiv  Detail & Related papers  (2025-03-12T15:46:12Z)
• Generating Unbiased Pseudo-labels via a Theoretically Guaranteed Chebyshev Constraint to Unify Semi-supervised Classification and Regression [57.17120203327993]
  threshold-to-pseudo label process (T2L) in classification uses confidence to determine the quality of label. In nature, regression also requires unbiased methods to generate high-quality labels. We propose a theoretically guaranteed constraint for generating unbiased labels based on Chebyshev's inequality.
  arXiv  Detail & Related papers  (2023-11-03T08:39:35Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• Auto-Encoding Goodness of Fit [9.560668678348579]
  We develop a new type of generative autoencoder called the Goodness-of-Fit Autoencoder (GoFAE)<n>At the minibatch level, it uses GoF test statistics as regularization objectives.<n>At a more global level, it selects a regularization coefficient based on higher criticism.
  arXiv  Detail & Related papers  (2022-10-12T19:21:57Z)
• Optimizing Partial Area Under the Top-k Curve: Theory and Practice [151.5072746015253]
  We develop a novel metric named partial Area Under the top-k Curve (AUTKC) AUTKC has a better discrimination ability, and its Bayes optimal score function could give a correct top-K ranking with respect to the conditional probability. We present an empirical surrogate risk minimization framework to optimize the proposed metric.
  arXiv  Detail & Related papers  (2022-09-03T11:09:13Z)
• A Unified Joint Maximum Mean Discrepancy for Domain Adaptation [73.44809425486767]
  This paper theoretically derives a unified form of JMMD that is easy to optimize. From the revealed unified JMMD, we illustrate that JMMD degrades the feature-label dependence that benefits to classification. We propose a novel MMD matrix to promote the dependence, and devise a novel label kernel that is robust to label distribution shift.
  arXiv  Detail & Related papers  (2021-01-25T09:46:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.