Class-wise Autoencoders Measure Classification Difficulty And Detect Label Mistakes

• URL: http://arxiv.org/abs/2412.02596v1
• Date: Tue, 03 Dec 2024 17:29:00 GMT
• Title: Class-wise Autoencoders Measure Classification Difficulty And Detect Label Mistakes
• Authors: Jacob Marks, Brent A. Griffin, Jason J. Corso,
• Abstract summary: We introduce a new framework for analyzing classification datasets based on the ratios of reconstruction errors between autoencoders trained on individual classes. This analysis framework enables efficient characterization of datasets on the sample, class, and entire dataset levels.
• Score: 22.45812577928658
• License:
• Abstract: We introduce a new framework for analyzing classification datasets based on the ratios of reconstruction errors between autoencoders trained on individual classes. This analysis framework enables efficient characterization of datasets on the sample, class, and entire dataset levels. We define reconstruction error ratios (RERs) that probe classification difficulty and allow its decomposition into (1) finite sample size and (2) Bayes error and decision-boundary complexity. Through systematic study across 19 popular visual datasets, we find that our RER-based dataset difficulty probe strongly correlates with error rate for state-of-the-art (SOTA) classification models. By interpreting sample-level classification difficulty as a label mistakenness score, we further find that RERs achieve SOTA performance on mislabel detection tasks on hard datasets under symmetric and asymmetric label noise. Our code is publicly available at https://github.com/voxel51/reconstruction-error-ratios.

Related papers

• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Semi-Supervised End-To-End Contrastive Learning For Time Series Classification [10.635321868623883]
  Time series classification is a critical task in various domains, such as finance, healthcare, and sensor data analysis. We propose an end-to-end model called SLOTS (Semi-supervised Learning fOr Time clasSification)
  arXiv  Detail & Related papers  (2023-10-13T04:22:21Z)
• End-to-End Supervised Multilabel Contrastive Learning [38.26579519598804]
  Multilabel representation learning is recognized as a challenging problem that can be associated with either label dependencies between object categories or data-related issues. Recent advances address these challenges from model- and data-centric viewpoints. We propose a new end-to-end training framework -- dubbed KMCL -- to address the shortcomings of both model- and data-centric designs.
  arXiv  Detail & Related papers  (2023-07-08T12:46:57Z)
• Learning Confident Classifiers in the Presence of Label Noise [5.551384206194696]
  This paper proposes a probabilistic model for noisy observations that allows us to build a confident classification and segmentation models. Our experiments show that our algorithm outperforms state-of-the-art solutions for the considered classification and segmentation problems.
  arXiv  Detail & Related papers  (2023-01-02T04:27:25Z)
• CTRL: Clustering Training Losses for Label Error Detection [4.49681473359251]
  In supervised machine learning, use of correct labels is extremely important to ensure high accuracy. We propose a novel framework, calledClustering TRaining Losses for label error detection. It detects label errors in two steps based on the observation that models learn clean and noisy labels in different ways.
  arXiv  Detail & Related papers  (2022-08-17T18:09:19Z)
• The Overlooked Classifier in Human-Object Interaction Recognition [82.20671129356037]
  We encode the semantic correlation among classes into the classification head by initializing the weights with language embeddings of HOIs. We propose a new loss named LSE-Sign to enhance multi-label learning on a long-tailed dataset. Our simple yet effective method enables detection-free HOI classification, outperforming the state-of-the-arts that require object detection and human pose by a clear margin.
  arXiv  Detail & Related papers  (2022-03-10T23:35:00Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
  In this paper we address the problem of classification in the presence of label noise. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space. Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
  arXiv  Detail & Related papers  (2021-11-22T15:49:20Z)
• Assessing the Quality of the Datasets by Identifying Mislabeled Samples [14.881597737762316]
  We propose a novel statistic -- noise score -- as a measure for the quality of each data point to identify mislabeled samples. In our work, we use the representations derived by the inference network of data quality supervised variational autoencoder (AQUAVS) We validate our proposed statistic through experimentation by corrupting MNIST, FashionMNIST, and CIFAR10/100 datasets.
  arXiv  Detail & Related papers  (2021-09-10T17:14:09Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)
• Improving Face Recognition by Clustering Unlabeled Faces in the Wild [77.48677160252198]
  We propose a novel identity separation method based on extreme value theory. It greatly reduces the problems caused by overlapping-identity label noise. Experiments on both controlled and real settings demonstrate our method's consistent improvements.
  arXiv  Detail & Related papers  (2020-07-14T12:26:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.