Deep Neural Networks with Ordinal Loss for Medical Applications

• URL: http://arxiv.org/abs/2606.25769v1
• Date: Wed, 24 Jun 2026 12:45:17 GMT
• Title: Deep Neural Networks with Ordinal Loss for Medical Applications
• Abstract summary: We introduce the textbfOrdinal Cross-Entropy (OCE) framework, a general and architecture-independent approach for learning from ordinal data.<n>OCE achieves lower prediction error costs and better calibration compared to existing state-of-the-art ordinal approaches.
• Score: 2.8496753993424817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In many prediction problems in medical applications, target labels exhibit an inherent ordinal structure, where class ordering reflects clinically meaningful severity levels. The cost associated with misclassification is often non-uniform and asymmetric, as errors between distant ordinal categories may have substantially more severe consequences than errors between adjacent ones, and overestimating disease severity may have different clinical implications than underestimating it. Traditional loss functions such as multi-class cross-entropy treat all misclassifications equally and fail to incorporate this ordering information. Recent advances in ordinal regression aim to address this limitation by integrating rank-based structures into deep learning models. In this work, we introduce the \textbf{Ordinal Cross-Entropy (OCE)} framework, a general and architecture-independent approach for learning from ordinal data. The proposed method extends the standard cross-entropy formulation to account for misclassification severity through an ordinal cost matrix while preserving the probabilistic interpretation and optimization benefits of the conventional loss. We provide a theoretical analysis of the OCE gradient behavior and show that it yields smoother optimization dynamics and improved ordinal consistency. Experiments on benchmark datasets show that our method achieves lower prediction error costs and better calibration compared to existing state-of-the-art ordinal approaches, establishing OCE as a simple yet effective solution for ordinal regression in deep neural networks.

Related papers

• Every Error has Its Magnitude: Asymmetric Mistake Severity Training for Multiclass Multiple Instance Learning [7.643169473987673]
  Multiple Instance Learning (MIL) has emerged as a promising paradigm for Whole Slide Image (WSI) diagnosis.<n>We propose a mistake-severity-aware training strategy that organizes diagnostic classes into a hierarchical structure.<n>We show that our approach significantly mitigates critical errors in MIL diagnosis compared to existing methods.
  arXiv  Detail & Related papers  (2026-03-14T01:36:12Z)
• An Orthogonal Learner for Individualized Outcomes in Markov Decision Processes [55.93922317950527]
  We develop a novel meta-learner called DRQ-learner.<n>Our DRQ-learner is applicable to settings with both discrete and continuous state spaces.
  arXiv  Detail & Related papers  (2025-09-30T15:49:29Z)
• Parameterized Diffusion Optimization enabled Autoregressive Ordinal Regression for Diabetic Retinopathy Grading [53.11883409422728]
  This work proposes a novel autoregressive ordinal regression method called AOR-DR.<n>We decompose the diabetic retinopathy grading task into a series of ordered steps by fusing the prediction of the previous steps with extracted image features.<n>We exploit the diffusion process to facilitate conditional probability modeling, enabling the direct use of continuous global image features for autoregression.
  arXiv  Detail & Related papers  (2025-07-07T13:22:35Z)
• NDCG-Consistent Softmax Approximation with Accelerated Convergence [67.10365329542365]
  We propose novel loss formulations that align directly with ranking metrics.<n>We integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method.<n> Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance.
  arXiv  Detail & Related papers  (2025-06-11T06:59:17Z)
• CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss [8.202961373604719]
  In ordinal classification, misclassifying neighboring ranks is common, yet the consequences of these errors are not the same.<n>We propose CLOC, a new margin-based contrastive learning method for ordinal classification.<n>CLOC learns an ordered representation based on the optimization of multiple margins with a novel multi-margin n-pair loss.
  arXiv  Detail & Related papers  (2025-04-22T22:23:30Z)
• Calibration of Ordinal Regression Networks [1.2242167538741824]
  Deep neural networks are not well-calibrated and often produce over-confident predictions.<n>We propose a novel loss function that introduces ordinal-aware calibration.<n>It incorporates soft ordinal encoding and ordinal-aware regularization to enforce both calibration and unimodality.
  arXiv  Detail & Related papers  (2024-10-21T05:56:31Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Continuously Generalized Ordinal Regression for Linear and Deep Models [41.03778663275373]
  Ordinal regression is a classification task where classes have an order and prediction error increases the further the predicted class is from the true class. We propose a new approach for modeling ordinal data that allows class-specific hyperplane slopes. Our method significantly outperforms the standard ordinal logistic model over a thorough set of ordinal regression benchmark datasets.
  arXiv  Detail & Related papers  (2022-02-14T19:49:05Z)
• Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation [1.957338076370071]
  We propose a novel loss function called class distance weighted cross-entropy (CDW-CE) that respects the order of the classes and takes the distance of the classes into account in calculation of cost. In this study, we propose a novel loss function called class distance weighted cross-entropy (CDW-CE) that respects the order of the classes and takes the distance of the classes into account in calculation of cost.
  arXiv  Detail & Related papers  (2022-02-09T18:47:50Z)
• Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities [3.093890460224435]
  Ordinal regression methods for deep neural networks address ordinal response variables. CORAL method achieves rank consistency among its output layer tasks by imposing a weight-sharing constraint. We propose a new method for rank-consistent ordinal regression without this limitation.
  arXiv  Detail & Related papers  (2021-11-17T01:10:23Z)
• CASTLE: Regularization via Auxiliary Causal Graph Discovery [89.74800176981842]
  We introduce Causal Structure Learning (CASTLE) regularization and propose to regularize a neural network by jointly learning the causal relationships between variables. CASTLE efficiently reconstructs only the features in the causal DAG that have a causal neighbor, whereas reconstruction-based regularizers suboptimally reconstruct all input features.
  arXiv  Detail & Related papers  (2020-09-28T09:49:38Z)
• Convolutional Ordinal Regression Forest for Image Ordinal Estimation [52.67784321853814]
  We propose a novel ordinal regression approach, termed Convolutional Ordinal Regression Forest or CORF, for image ordinal estimation. The proposed CORF integrates ordinal regression and differentiable decision trees with a convolutional neural network for obtaining precise and stable global ordinal relationships. The effectiveness of the proposed CORF is verified on two image ordinal estimation tasks, showing significant improvements and better stability over the state-of-the-art ordinal regression methods.
  arXiv  Detail & Related papers  (2020-08-07T10:41:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.