When Labels Have Structure: Improving Image Classification with Hierarchy-Aware Cross-Entropy

• URL: http://arxiv.org/abs/2605.06274v1
• Date: Thu, 07 May 2026 13:49:36 GMT
• Title: When Labels Have Structure: Improving Image Classification with Hierarchy-Aware Cross-Entropy
• Abstract summary: HACE is a drop-in replacement for standard cross-entropy that incorporates a known class hierarchy directly into the loss.<n>In end-to-end training, HACE improves accuracy over standard cross-entropy in 15 out of 18 architecture--dataset pairs.<n>In linear probing on frozen DINOv2-Large features, HACE outperforms all competing methods on all datasets.
• Score: 0.09558392439655013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Standard cross-entropy is the default classification loss across virtually all of machine learning, yet it treats all misclassifications equally, ignoring the semantic distances that a class hierarchy encodes. We propose Hierarchy-Aware Cross-Entropy (HACE), a drop-in replacement for standard cross-entropy that incorporates a known class hierarchy directly into the loss. HACE combines two components: prediction aggregation, which propagates the model's probability mass upward through the class hierarchy to ensure that parent nodes accumulate the confidence of their children; and ancestral label smoothing, which distributes the ground-truth signal along the path from the true class to the root. We evaluate HACE on CIFAR-100, FGVC Aircraft, and NABirds in two regimes: end-to-end training across six architectures spanning convolutional and attention-based designs, and linear probing on frozen DINOv2-Large features. In end-to-end training, HACE improves accuracy over standard cross-entropy in 15 out of 18 architecture--dataset pairs, with a mean gain of 4.66\%. In linear probing on frozen DINOv2-Large features, HACE outperforms all competing methods on all three datasets, with a mean improvement of 2.18\% over the next best baseline.

Related papers

• Hyperbolic Latent Geometry for Tree-Structured Prototype Networks: A Local-vs-Global Trade-off [0.0]
  We study a tree-structured regularizer over class-prototype layouts in a hierarchical-classification model.<n>We find that Poincare prototypes preserve the topology of the nearest-neighbor graph in latent space substantially better than matched Euclidean prototypes.
  arXiv  Detail & Related papers  (2026-08-25T22:42:20Z)
• Online Continual Learning with Dynamic Label Hierarchies [72.33335166136045]
  We introduce a new problem setting, DHOCL, where severity evolve across granularities and each sample provides supervision at a single hierarchical level.<n>In this setting, we find two fundamental issues: (i) partial supervision under mixed granularities provides only point-wise signals over an evolving path-wise hierarchy, which constrains plasticity and undermines cross-level semantic consistency.<n>We propose HALO, which adaptively combines complementary classification heads, regularized by organized learnable hierarchical prototypes, enabling rapid adaptation, hierarchical consistency, and structured knowledge consolidation as the taxonomy evolves.
  arXiv  Detail & Related papers  (2026-05-12T08:20:23Z)
• CrossHGL: A Text-Free Foundation Model for Cross-Domain Heterogeneous Graph Learning [18.84875776859183]
  CrossHGL is a semantic-preserving transformation strategy that homogenizes heterogeneous graphs and encodes interaction semantics into edge features.<n>A prompt-aware multi-domain pre-training framework with a Tri-Prompt mechanism captures transferable knowledge across feature, edge, and structure perspectives.<n>Experiments on node-level and graph-level tasks show that CrossHGL consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-29T13:17:15Z)
• FUSE: Fast Semi-Supervised Node Embedding Learning via Structural and Label-Aware Optimization [0.0]
  We introduce a fast semi-supervised embedding framework that jointly optimize three complementary objectives.<n>On standard benchmarks, our method consistently achieves classification accuracy at par with or superior to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-10-13T10:39:58Z)
• AHDMIL: Asymmetric Hierarchical Distillation Multi-Instance Learning for Fast and Accurate Whole-Slide Image Classification [51.525891360380285]
  AHDMIL is an Asymmetric Hierarchical Distillation Multi-Instance Learning framework.<n>It eliminates irrelevant patches through a two-step training process.<n>It consistently outperforms previous state-of-the-art methods in both classification performance and inference speed.
  arXiv  Detail & Related papers  (2025-08-07T07:47:16Z)
• Cross-architecture universal feature coding via distribution alignment [88.73189953617594]
  We introduce a new research problem: cross-architecture universal feature coding (CAUFC)<n>We propose a two-step distribution alignment method. First, we design the format alignment method that CNN and Transformer features into a consistent 2D token format. Second, we propose the feature value alignment method that harmonizes statistical distributions via truncation and normalization.<n>As a first attempt to study CAUFC, we evaluate our method on the image classification task. Experimental results demonstrate that our method achieves superior rate-accuracy trade-offs compared to the architecture-specific baseline.
  arXiv  Detail & Related papers  (2025-06-15T06:14:02Z)
• Learning Robust Heterogeneous Graph Representations via Contrastive-Reconstruction under Sparse Semantics [13.555683316315683]
  Masked autoencoders (MAE) and contrastive learning (CL) are two prominent paradigms in graph self-supervised learning.<n>This paper introduces HetCRF, a novel dual-channel self-supervised learning framework for heterogeneous graphs.<n>HetCRF uses a two-stage aggregation strategy to adapt embedding semantics, making it compatible with both MAE and CL.
  arXiv  Detail & Related papers  (2025-06-07T06:35:42Z)
• Learning and Evaluating Hierarchical Feature Representations [3.770103075126785]
  We propose a novel framework, Hierarchical Composition of Orthogonal Subspaces (Hier-COS)<n>Hier-COS learns to map deep feature embeddings into a vector space that is, by design, consistent with the structure of a given taxonomy tree.<n>We demonstrate that Hier-COS achieves state-of-the-art hierarchical performance across all the datasets while simultaneously beating top-1 accuracy in all but one case.
  arXiv  Detail & Related papers  (2025-03-10T20:59:41Z)
• Learning Hierarchy Aware Features for Reducing Mistake Severity [3.704832909610283]
  We propose a novel approach for learning hierarchy aware features (HAF) HAF is a training time approach that improves the mistakes while maintaining top-1 error, thereby, addressing the problem of cross-entropy loss that treats all mistakes as equal. We evaluate HAF on three hierarchical datasets and achieve state-of-the-art results on the iNaturalist-19 and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2022-07-26T04:24:47Z)
• GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning [55.79997930181418]
  Generalized Zero-Shot Learning aims to recognize images from both the seen and unseen classes by transferring semantic knowledge from seen to unseen classes. It is a promising solution to take the advantage of generative models to hallucinate realistic unseen samples based on the knowledge learned from the seen classes. We propose a novel flow-based generative framework that consists of multiple conditional affine coupling layers for learning unseen data generation.
  arXiv  Detail & Related papers  (2022-07-05T04:04:37Z)
• Do We Really Need a Learnable Classifier at the End of Deep Neural Network? [118.18554882199676]
  We study the potential of learning a neural network for classification with the classifier randomly as an ETF and fixed during training. Our experimental results show that our method is able to achieve similar performances on image classification for balanced datasets.
  arXiv  Detail & Related papers  (2022-03-17T04:34:28Z)
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity [55.29408396918968]
  We study a family of loss functions named label-distributionally robust (LDR) losses for multi-class classification. Our contributions include both consistency and robustness by establishing top-$k$ consistency of LDR losses for multi-class classification. We propose a new adaptive LDR loss that automatically adapts the individualized temperature parameter to the noise degree of class label of each instance.
  arXiv  Detail & Related papers  (2021-12-30T00:27:30Z)
• Heuristic Semi-Supervised Learning for Graph Generation Inspired by Electoral College [80.67842220664231]
  We propose a novel pre-processing technique, namely ELectoral COllege (ELCO), which automatically expands new nodes and edges to refine the label similarity within a dense subgraph. In all setups tested, our method boosts the average score of base models by a large margin of 4.7 points, as well as consistently outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2020-06-10T14:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.