MLMC: Interactive multi-label multi-classifier evaluation without confusion matrices

• URL: http://arxiv.org/abs/2501.14460v1
• Date: Fri, 24 Jan 2025 12:43:36 GMT
• Title: MLMC: Interactive multi-label multi-classifier evaluation without confusion matrices
• Authors: Aleksandar Doknic, Torsten Möller,
• Abstract summary: Machine-C is a visual exploration tool that tackles the challenge of multi-label comparison and evaluation. Our study shows that the techniques implemented by Machine-C allow for a powerful multi-label classifier evaluation while preserving user friendliness.
• Score: 52.476815843373515
• License:
• Abstract: Machine learning-based classifiers are commonly evaluated by metrics like accuracy, but deeper analysis is required to understand their strengths and weaknesses. MLMC is a visual exploration tool that tackles the challenge of multi-label classifier comparison and evaluation. It offers a scalable alternative to confusion matrices which are commonly used for such tasks, but don't scale well with a large number of classes or labels. Additionally, MLMC allows users to view classifier performance from an instance perspective, a label perspective, and a classifier perspective. Our user study shows that the techniques implemented by MLMC allow for a powerful multi-label classifier evaluation while preserving user friendliness.

Related papers

• Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models [31.34575955517015]
  Finedefics is an MLLM that enhances the model's FGVR capability by incorporating informative attribute descriptions of objects into the training phase. We employ contrastive learning on object-attribute pairs and attribute-category pairs simultaneously and use examples from similar but incorrect categories as hard negatives. Extensive evaluations across multiple popular FGVR datasets demonstrate that Finedefics outperforms existing MLLMs of comparable parameter sizes.
  arXiv  Detail & Related papers  (2025-01-25T08:52:43Z)
• Disambiguated Attention Embedding for Multi-Instance Partial-Label Learning [68.56193228008466]
  In many real-world tasks, the concerned objects can be represented as a multi-instance bag associated with a candidate label set. Existing MIPL approach follows the instance-space paradigm by assigning augmented candidate label sets of bags to each instance and aggregating bag-level labels from instance-level labels. We propose an intuitive algorithm named DEMIPL, i.e., Disambiguated attention Embedding for Multi-Instance Partial-Label learning.
  arXiv  Detail & Related papers  (2023-05-26T13:25:17Z)
• Reliable Representations Learning for Incomplete Multi-View Partial Multi-Label Classification [78.15629210659516]
  In this paper, we propose an incomplete multi-view partial multi-label classification network named RANK. We break through the view-level weights inherent in existing methods and propose a quality-aware sub-network to dynamically assign quality scores to each view of each sample. Our model is not only able to handle complete multi-view multi-label datasets, but also works on datasets with missing instances and labels.
  arXiv  Detail & Related papers  (2023-03-30T03:09:25Z)
• Towards Learning Causal Representations from Multi-Instance Bags [38.68813959335736]
  Multi-instance learning (MIL) is a type of weakly supervised learning that deals with objects represented as groups of instances. We propose the TargetedMIL algorithm, which learns semantically meaningful representations that can be interpreted as causal to the object of interest.
  arXiv  Detail & Related papers  (2022-02-25T09:13:00Z)
• Adaptive Hierarchical Similarity Metric Learning with Noisy Labels [138.41576366096137]
  We propose an Adaptive Hierarchical Similarity Metric Learning method. It considers two noise-insensitive information, textiti.e., class-wise divergence and sample-wise consistency. Our method achieves state-of-the-art performance compared with current deep metric learning approaches.
  arXiv  Detail & Related papers  (2021-10-29T02:12:18Z)
• Learning from Multiple Annotators by Incorporating Instance Features [15.643325526074804]
  Learning from multiple annotators aims to induce a high-quality classifier from training instances. Most existing methods adopt class-level confusion matrices of annotators that observed labels do not depend on the instance features. We propose a noise transition matrix, which incorporates the influence of instance features on annotators' performance based on confusion matrices.
  arXiv  Detail & Related papers  (2021-06-29T08:07:24Z)
• Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning [91.58529629419135]
  We consider how to characterise visual groupings discovered automatically by deep neural networks. We introduce two concepts, visual learnability and describability, that can be used to quantify the interpretability of arbitrary image groupings.
  arXiv  Detail & Related papers  (2020-10-27T18:41:49Z)
• Metrics for Multi-Class Classification: an Overview [0.9176056742068814]
  Classification tasks involving more than two classes are known as "multi-class classification" Performance indicators are very useful when the aim is to evaluate and compare different classification models or machine learning techniques.
  arXiv  Detail & Related papers  (2020-08-13T08:41:44Z)
• Decoding machine learning benchmarks [0.0]
  Item Response Theory (IRT) has emerged as a new approach to elucidate what should be a good machine learning benchmark. IRT was applied to explore the well-known OpenML-CC18 benchmark to identify how suitable it is on the evaluation of classifiers.
  arXiv  Detail & Related papers  (2020-07-29T14:39:41Z)
• Interaction Matching for Long-Tail Multi-Label Classification [57.262792333593644]
  We present an elegant and effective approach for addressing limitations in existing multi-label classification models. By performing soft n-gram interaction matching, we match labels with natural language descriptions.
  arXiv  Detail & Related papers  (2020-05-18T15:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.