To Each Metric Its Decoding: Post-Hoc Optimal Decision Rules of Probabilistic Hierarchical Classifiers

• URL: http://arxiv.org/abs/2506.01552v1
• Date: Mon, 02 Jun 2025 11:29:40 GMT
• Title: To Each Metric Its Decoding: Post-Hoc Optimal Decision Rules of Probabilistic Hierarchical Classifiers
• Authors: Roman Plaud, Alexandre Perez-Lebel, Matthieu Labeau, Antoine Saillenfest, Thomas Bonald,
• Abstract summary: We propose a framework for the optimal decoding of an output probability distribution with respect to a target metric.<n>We derive optimal decision rules for increasingly complex prediction settings, providing universal algorithms when candidates are limited to the set of nodes.
• Score: 43.97690773039761
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Hierarchical classification offers an approach to incorporate the concept of mistake severity by leveraging a structured, labeled hierarchy. However, decoding in such settings frequently relies on heuristic decision rules, which may not align with task-specific evaluation metrics. In this work, we propose a framework for the optimal decoding of an output probability distribution with respect to a target metric. We derive optimal decision rules for increasingly complex prediction settings, providing universal algorithms when candidates are limited to the set of nodes. In the most general case of predicting a subset of nodes, we focus on rules dedicated to the hierarchical $hF_{\beta}$ scores, tailored to hierarchical settings. To demonstrate the practical utility of our approach, we conduct extensive empirical evaluations, showcasing the superiority of our proposed optimal strategies, particularly in underdetermined scenarios. These results highlight the potential of our methods to enhance the performance and reliability of hierarchical classifiers in real-world applications. The code is available at https://github.com/RomanPlaud/hierarchical_decision_rules

Related papers

• C-3DPO: Constrained Controlled Classification for Direct Preference Optimization [23.709526350060816]
  Direct preference optimization (DPO)-style algorithms have emerged as a promising approach for solving the alignment problem in AI.<n>We present a novel perspective that formulates these algorithms as implicit classification algorithms.<n>We then leverage this classification framework to demonstrate that the underlying problem solved in these algorithms is under-specified.
  arXiv  Detail & Related papers  (2025-02-22T00:38:44Z)
• Conformal Prediction in Hierarchical Classification [18.730305100193927]
  We extend the split conformal prediction framework to hierarchical classification, where prediction sets are commonly restricted to internal nodes of a predefined hierarchy.<n>The first algorithm returns internal nodes as prediction sets, while the second relaxes this restriction, using the notion of complexity.<n> Empirical evaluations on several benchmark datasets demonstrate the effectiveness of the proposed algorithms.
  arXiv  Detail & Related papers  (2025-01-31T11:10:19Z)
• An incremental preference elicitation-based approach to learning potentially non-monotonic preferences in multi-criteria sorting [53.36437745983783]
  We first construct a max-margin optimization-based model to model potentially non-monotonic preferences. We devise information amount measurement methods and question selection strategies to pinpoint the most informative alternative in each iteration. Two incremental preference elicitation-based algorithms are developed to learn potentially non-monotonic preferences.
  arXiv  Detail & Related papers  (2024-09-04T14:36:20Z)
• Hierarchical Selective Classification [17.136832159667204]
  This paper introduces hierarchical selective classification, extending selective classification to a hierarchical setting.<n>We first formalize hierarchical risk and coverage, and introduce hierarchical risk-coverage curves.<n>Next, we develop algorithms for hierarchical selective classification, and propose an efficient algorithm that guarantees a target accuracy constraint with high probability.
  arXiv  Detail & Related papers  (2024-05-19T12:24:30Z)
• Optimal Baseline Corrections for Off-Policy Contextual Bandits [61.740094604552475]
  We aim to learn decision policies that optimize an unbiased offline estimate of an online reward metric. We propose a single framework built on their equivalence in learning scenarios. Our framework enables us to characterize the variance-optimal unbiased estimator and provide a closed-form solution for it.
  arXiv  Detail & Related papers  (2024-05-09T12:52:22Z)
• Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systems [33.46891569350896]
  Cascade ranking is widely used for large-scale top-k selection problems in online advertising and recommendation systems. Previous works on learning-to-rank usually focus on letting the model learn the complete order or top-k order. We name this method as Adaptive Neural Ranking Framework (abbreviated as ARF)
  arXiv  Detail & Related papers  (2023-10-16T14:43:02Z)
• Multi-Task Off-Policy Learning from Bandit Feedback [54.96011624223482]
  We propose a hierarchical off-policy optimization algorithm (HierOPO), which estimates the parameters of the hierarchical model and then acts pessimistically with respect to them. We prove per-task bounds on the suboptimality of the learned policies, which show a clear improvement over not using the hierarchical model. Our theoretical and empirical results show a clear advantage of using the hierarchy over solving each task independently.
  arXiv  Detail & Related papers  (2022-12-09T08:26:27Z)
• Hierarchical classification at multiple operating points [1.520694326234112]
  We present an efficient algorithm to produce operating characteristic curves for any method that assigns a score to every class in the hierarchy. We propose two novel loss functions and show that a soft variant of the structured hinge loss is able to significantly outperform the flat baseline.
  arXiv  Detail & Related papers  (2022-10-19T23:36:16Z)
• Weakly-supervised Action Localization via Hierarchical Mining [76.00021423700497]
  Weakly-supervised action localization aims to localize and classify action instances in the given videos temporally with only video-level categorical labels. We propose a hierarchical mining strategy under video-level and snippet-level manners, i.e., hierarchical supervision and hierarchical consistency mining. We show that HiM-Net outperforms existing methods on THUMOS14 and ActivityNet1.3 datasets with large margins by hierarchically mining the supervision and consistency.
  arXiv  Detail & Related papers  (2022-06-22T12:19:09Z)
• On the Optimality of Batch Policy Optimization Algorithms [106.89498352537682]
  Batch policy optimization considers leveraging existing data for policy construction before interacting with an environment. We show that any confidence-adjusted index algorithm is minimax optimal, whether it be optimistic, pessimistic or neutral. We introduce a new weighted-minimax criterion that considers the inherent difficulty of optimal value prediction.
  arXiv  Detail & Related papers  (2021-04-06T05:23:20Z)
• Stochastic batch size for adaptive regularization in deep network optimization [63.68104397173262]
  We propose a first-order optimization algorithm incorporating adaptive regularization applicable to machine learning problems in deep learning framework. We empirically demonstrate the effectiveness of our algorithm using an image classification task based on conventional network models applied to commonly used benchmark datasets.
  arXiv  Detail & Related papers  (2020-04-14T07:54:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.