Hierarchical confusion matrix for classification performance evaluation

• URL: http://arxiv.org/abs/2306.09461v1
• Date: Thu, 15 Jun 2023 19:31:59 GMT
• Title: Hierarchical confusion matrix for classification performance evaluation
• Authors: Kevin Riehl, Michael Neunteufel, Martin Hemberg
• Abstract summary: We develop the concept of a hierarchical confusion matrix and prove its applicability to all types of hierarchical classification problems. We use measures based on the novel confusion matrix to evaluate models within a benchmark for three real world hierarchical classification applications. The results outline the reasonability of this approach and its usefulness to evaluate hierarchical classification problems.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work we propose a novel concept of a hierarchical confusion matrix, opening the door for popular confusion matrix based (flat) evaluation measures from binary classification problems, while considering the peculiarities of hierarchical classification problems. We develop the concept to a generalized form and prove its applicability to all types of hierarchical classification problems including directed acyclic graphs, multi path labelling, and non mandatory leaf node prediction. Finally, we use measures based on the novel confusion matrix to evaluate models within a benchmark for three real world hierarchical classification applications and compare the results to established evaluation measures. The results outline the reasonability of this approach and its usefulness to evaluate hierarchical classification problems. The implementation of hierarchical confusion matrix is available on GitHub.

Related papers

• Reducing the dimensionality and granularity in hierarchical categorical variables [2.089191490381739]
  We propose a methodology to obtain a reduced representation of a hierarchical categorical variable. We show how entity embedding can be applied in a hierarchical setting. We apply our methodology on a real dataset and find that the reduced hierarchy is an improvement over the original hierarchical structure.
  arXiv  Detail & Related papers  (2024-03-06T11:09:36Z)
• Deep Imbalanced Regression via Hierarchical Classification Adjustment [50.19438850112964]
  Regression tasks in computer vision are often formulated into classification by quantizing the target space into classes. The majority of training samples lie in a head range of target values, while a minority of samples span a usually larger tail range. We propose to construct hierarchical classifiers for solving imbalanced regression tasks. Our novel hierarchical classification adjustment (HCA) for imbalanced regression shows superior results on three diverse tasks.
  arXiv  Detail & Related papers  (2023-10-26T04:54:39Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• A Systematic Evaluation of Node Embedding Robustness [77.29026280120277]
  We assess the empirical robustness of node embedding models to random and adversarial poisoning attacks. We compare edge addition, deletion and rewiring strategies computed using network properties as well as node labels. We found that node classification suffers from higher performance degradation as opposed to network reconstruction.
  arXiv  Detail & Related papers  (2022-09-16T17:20:23Z)
• Weakly-supervised Action Localization via Hierarchical Mining [76.00021423700497]
  Weakly-supervised action localization aims to localize and classify action instances in the given videos temporally with only video-level categorical labels. We propose a hierarchical mining strategy under video-level and snippet-level manners, i.e., hierarchical supervision and hierarchical consistency mining. We show that HiM-Net outperforms existing methods on THUMOS14 and ActivityNet1.3 datasets with large margins by hierarchically mining the supervision and consistency.
  arXiv  Detail & Related papers  (2022-06-22T12:19:09Z)
• A Top-down Supervised Learning Approach to Hierarchical Multi-label Classification in Networks [0.21485350418225244]
  This paper presents a general prediction model to hierarchical multi-label classification (HMC), where the attributes to be inferred can be specified as a strict poset. It is based on a top-down classification approach that addresses hierarchical multi-label classification with supervised learning by building a local classifier per class. The proposed model is showcased with a case study on the prediction of gene functions for Oryza sativa Japonica, a variety of rice.
  arXiv  Detail & Related papers  (2022-03-23T17:29:17Z)
• Inducing a hierarchy for multi-class classification problems [11.58041597483471]
  In applications where categorical labels follow a natural hierarchy, classification methods that exploit the label structure often outperform those that do not. In this paper, we investigate a class of methods that induce a hierarchy that can similarly improve classification performance over flat classifiers. We demonstrate the effectiveness of the class of methods both for discovering a latent hierarchy and for improving accuracy in principled simulation settings and three real data applications.
  arXiv  Detail & Related papers  (2021-02-20T05:40:42Z)
• Pitfalls of Assessing Extracted Hierarchies for Multi-Class Classification [4.89253144446913]
  We identify some common pitfalls that may lead practitioners to make misleading conclusions about their methods. We show how the hierarchy's quality can become irrelevant depending on the experimental setup. Our results confirm that datasets with a high number of classes generally present complex structures in how these classes relate to each other.
  arXiv  Detail & Related papers  (2021-01-26T21:50:57Z)
• Coherent Hierarchical Multi-Label Classification Networks [56.41950277906307]
  C-HMCNN(h) is a novel approach for HMC problems, which exploits hierarchy information in order to produce predictions coherent with the constraint and improve performance. We conduct an extensive experimental analysis showing the superior performance of C-HMCNN(h) when compared to state-of-the-art models.
  arXiv  Detail & Related papers  (2020-10-20T09:37:02Z)
• Structured Graph Learning for Clustering and Semi-supervised Classification [74.35376212789132]
  We propose a graph learning framework to preserve both the local and global structure of data. Our method uses the self-expressiveness of samples to capture the global structure and adaptive neighbor approach to respect the local structure. Our model is equivalent to a combination of kernel k-means and k-means methods under certain condition.
  arXiv  Detail & Related papers  (2020-08-31T08:41:20Z)
• Deep Hierarchical Classification for Category Prediction in E-commerce System [16.6932395109085]
  In e-commerce system, category prediction is to automatically predict categories of given texts. We propose a Deep Hierarchical Classification framework, which incorporates the multi-scale hierarchical information in neural networks. We also define a novel combined loss function to punish hierarchical prediction losses.
  arXiv  Detail & Related papers  (2020-05-14T02:29:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.