D3O: Dynamic Distribution Distillation for Ordinal Regression

• URL: http://arxiv.org/abs/2607.23575v1
• Date: Sun, 26 Jul 2026 09:58:04 GMT
• Title: D3O: Dynamic Distribution Distillation for Ordinal Regression
• Abstract summary: Ordinal regression is widely used in scenarios where labels are discrete yet inherently ordered.<n>We propose D3O, a dynamic distribution distillation framework that replaces static supervision with training-driven evolution of ordinal label distributions.
• Score: 9.784209833489166
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Ordinal regression is widely used in scenarios where labels are discrete yet inherently ordered. In practice, however, ordinal labels are often obtained by discretizing underlying continuous semantics through subjective human judgment, resulting in ambiguous boundaries and annotation noise. Such uncertainty challenges existing methods that rely on fixed supervision targets, which may reinforce biased ordering under subjective annotations. To address this limitation, we propose D3O, a dynamic distribution distillation framework that replaces static supervision with training-driven evolution of ordinal label distributions via self-distillation. Specifically, we introduce a contrastive ordinal-aware label enhancement module that leverages vision-language alignment to recover refined label distributions capturing both inter-class ambiguity and instance-level uncertainty. Furthermore, we design a CDF-based cross-layer interaction distillation mechanism to propagate cumulative ordinal structure across network hierarchy, ensuring consistent ordinal geometry in intermediate representations. Extensive experiments on four general ordinal regression tasks demonstrate that our proposed D3O consistently outperforms existing approaches, particularly under severe class imbalance and noisy supervision. These results highlight the effectiveness of dynamic supervision in learning robust ordinal representations beyond fixed targets. The code will be publicly available.

Related papers

• DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation [58.47973015036709]
  DisPOSE is a self-supervised framework that approximates the inherently discrete multi-view person-assignment problem.<n>By employing differentiable Sinkhorn projections during denoising, our model learns to guide solutions toward valid and feasible assignments.<n>The proposed approach outperforms current state-of-the-art self-supervised methods on standard datasets.
  arXiv  Detail & Related papers  (2026-06-05T16:14:39Z)
• DiffoR: A Unified Continuous Generative Framework for Universal Ordinal Regression [37.354096693649375]
  Ordinal Regression aims to predict target values with inherent order.<n>Existing paradigms are constrained by quantization artifacts and the lack of global ordinal topological perception.<n>Under the novel paradigm, we introduce DiffOR, a unified framework that leverages diffusion models to recover continuous ordinal values.
  arXiv  Detail & Related papers  (2026-05-29T07:38:39Z)
• Learning Transferable Predictability Representations [2.1505812656718795]
  We study the problem of assigning a scalar score to a short trajectory window that reflects its position on an ordered continuum of predictability regimes.<n>We formalize this as ordinal estimation over a five-level predictability ladder and identify a structural source of cross-system ambiguity.<n>We propose the Gauge-Fixed Ordinal Network (GON), a temporal convolutional model trained with an anchor-and-variance objective that pins level-wise score means to shared target coordinates.
  arXiv  Detail & Related papers  (2026-05-28T21:38:49Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Explicit Dropout: Deterministic Regularization for Transformer Architectures [55.09895958546215]
  Dropout is a widely used regularization technique in deep learning, but its effects are typically realized through masking.<n>We propose a deterministic formulation that expresses dropout as an additive regularizer directly incorporated into the training loss.
  arXiv  Detail & Related papers  (2026-04-22T12:45:51Z)
• The Emergence of Lab-Driven Alignment Signatures: A Psychometric Framework for Auditing Latent Bias and Compounding Risk in Generative AI [0.0]
  This paper introduces a novel auditing framework to quantify latent trait estimation under ordinal uncertainty.<n>The research audits nine leading models across dimensions including Optimization Bias, Sycophancy, and Status-Quo Legitimization.
  arXiv  Detail & Related papers  (2026-02-19T06:56:01Z)
• CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation [67.36817440834251]
  We propose a unified framework for textbfCLass-incremental textbfImbalance-aware textbf3DIS.<n>Our approach achieves state-of-the-art results, surpassing prior work by up to 16.76% mAP for instance segmentation and approximately 30% mIoU for semantic segmentation.
  arXiv  Detail & Related papers  (2025-02-24T18:58:58Z)
• Unsupervised Domain Adaptation for Action Recognition via Self-Ensembling and Conditional Embedding Alignment [2.06242362470764]
  We propose a novel joint optimization architecture comprised of three functions: consistency regularizer, temporal ensemble and conditional distribution alignment. $mu$DAR results in a range of $approx$ 4-12% average macro-F1 score improvement over six state-of-the-art UDA methods in four benchmark wHAR datasets.
  arXiv  Detail & Related papers  (2024-10-23T00:59:27Z)
• CORE: Learning Consistent Ordinal REpresentations for Image Ordinal Estimation [35.39143939072549]
  This paper proposes learning intrinsic Consistent Ordinal REpresentations (CORE) from ordinal relations residing in groundtruth labels. CORE can accurately construct an ordinal latent space and significantly enhance existing deep ordinal regression methods to achieve better results.
  arXiv  Detail & Related papers  (2023-01-15T15:42:26Z)
• Exploiting Completeness and Uncertainty of Pseudo Labels for Weakly Supervised Video Anomaly Detection [149.23913018423022]
  Weakly supervised video anomaly detection aims to identify abnormal events in videos using only video-level labels. Two-stage self-training methods have achieved significant improvements by self-generating pseudo labels. We propose an enhancement framework by exploiting completeness and uncertainty properties for effective self-training.
  arXiv  Detail & Related papers  (2022-12-08T05:53:53Z)
• Self-supervised debiasing using low rank regularization [59.84695042540525]
  Spurious correlations can cause strong biases in deep neural networks, impairing generalization ability. We propose a self-supervised debiasing framework potentially compatible with unlabeled samples. Remarkably, the proposed debiasing framework significantly improves the generalization performance of self-supervised learning baselines.
  arXiv  Detail & Related papers  (2022-10-11T08:26:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.