Hierarchical Conformal Classification

• URL: http://arxiv.org/abs/2508.13288v1
• Date: Mon, 18 Aug 2025 18:05:55 GMT
• Title: Hierarchical Conformal Classification
• Authors: Floris den Hengst, Inès Blin, Majid Mohammadi, Syed Ihtesham Hussain Shah, Taraneh Younesian,
• Abstract summary: Conformal prediction (CP) is a powerful framework for quantifying uncertainty in machine learning models.<n>Standard CP treats classes as flat and unstructured, ignoring relationships such as semantic or hierarchical structure among class labels.<n>This paper presents hierarchical HCCal classification (HCC), an extension of CP that incorporates class hierarchies into both the structure and semantics of prediction sets.
• Score: 5.964388602612373
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Conformal prediction (CP) is a powerful framework for quantifying uncertainty in machine learning models, offering reliable predictions with finite-sample coverage guarantees. When applied to classification, CP produces a prediction set of possible labels that is guaranteed to contain the true label with high probability, regardless of the underlying classifier. However, standard CP treats classes as flat and unstructured, ignoring domain knowledge such as semantic relationships or hierarchical structure among class labels. This paper presents hierarchical conformal classification (HCC), an extension of CP that incorporates class hierarchies into both the structure and semantics of prediction sets. We formulate HCC as a constrained optimization problem whose solutions yield prediction sets composed of nodes at different levels of the hierarchy, while maintaining coverage guarantees. To address the combinatorial nature of the problem, we formally show that a much smaller, well-structured subset of candidate solutions suffices to ensure coverage while upholding optimality. An empirical evaluation on three new benchmarks consisting of audio, image, and text data highlights the advantages of our approach, and a user study shows that annotators significantly prefer hierarchical over flat prediction sets.

Related papers

• Reliable Hierarchical Operating System Fingerprinting via Conformal Prediction [62.40452053128524]
  Conformal Prediction (CP) could be wrapped around existing methods to obtain prediction sets with guaranteed coverage.<n>This work addresses these limitations by introducing and evaluating two distinct structured CP strategies.<n>While both methods satisfy validity guarantees, they expose a fundamental trade-off between level-wise efficiency and structural consistency.
  arXiv  Detail & Related papers  (2026-02-13T11:20:48Z)
• Enforcing Consistency and Fairness in Multi-level Hierarchical Classification with a Mask-based Output Layer [25.819440955594736]
  We introduce a fair, model-agnostic layer designed to enforce taxonomy and optimize objectives, including consistency, fairness, and exact match.<n>Our evaluations demonstrate that the proposed layer not only improves the fairness of predictions but also enforces the taxonomy, resulting in consistent predictions and superior performance.
  arXiv  Detail & Related papers  (2025-03-19T06:30:04Z)
• Conformal Prediction Sets with Improved Conditional Coverage using Trust Scores [52.92618442300405]
  It is impossible to achieve exact, distribution-free conditional coverage in finite samples.<n>We propose an alternative conformal prediction algorithm that targets coverage where it matters most.
  arXiv  Detail & Related papers  (2025-01-17T12:01:56Z)
• Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration [30.039997048267]
  This paper proposes the Rank Calibrated Class-conditional CP (RC3P) algorithm to reduce the prediction set sizes to achieve class-conditional coverage.<n>Experiments on multiple real-world datasets demonstrate that RC3P achieves class-wise coverage and 26.25% reduction in prediction set sizes on average.
  arXiv  Detail & Related papers  (2024-06-10T22:01:34Z)
• Hierarchical Selective Classification [17.136832159667204]
  This paper introduces hierarchical selective classification, extending selective classification to a hierarchical setting.<n>We first formalize hierarchical risk and coverage, and introduce hierarchical risk-coverage curves.<n>Next, we develop algorithms for hierarchical selective classification, and propose an efficient algorithm that guarantees a target accuracy constraint with high probability.
  arXiv  Detail & Related papers  (2024-05-19T12:24:30Z)
• Class-Conditional Conformal Prediction with Many Classes [60.8189977620604]
  We propose a method called clustered conformal prediction that clusters together classes having "similar" conformal scores. We find that clustered conformal typically outperforms existing methods in terms of class-conditional coverage and set size metrics.
  arXiv  Detail & Related papers  (2023-06-15T17:59:02Z)
• ProTeCt: Prompt Tuning for Taxonomic Open Set Classification [59.59442518849203]
  Few-shot adaptation methods do not fare well in the taxonomic open set (TOS) setting. We propose a prompt tuning technique that calibrates the hierarchical consistency of model predictions. A new Prompt Tuning for Hierarchical Consistency (ProTeCt) technique is then proposed to calibrate classification across label set granularities.
  arXiv  Detail & Related papers  (2023-06-04T02:55:25Z)
• Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification [94.55805516167369]
  We propose a new approach for binary classification from m U-sets for $mge2$. Our key idea is to consider an auxiliary classification task called surrogate set classification (SSC)
  arXiv  Detail & Related papers  (2021-02-01T07:36:38Z)
• Selective Classification via One-Sided Prediction [54.05407231648068]
  One-sided prediction (OSP) based relaxation yields an SC scheme that attains near-optimal coverage in the practically relevant high target accuracy regime. We theoretically derive bounds generalization for SC and OSP, and empirically we show that our scheme strongly outperforms state of the art methods in coverage at small error levels.
  arXiv  Detail & Related papers  (2020-10-15T16:14:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.