A Novel Method to Evaluate Models on Unreliable, Noisy and Inconsistent Labels: Adaptive Resolution Label Aggregation (ARLA)

• URL: http://arxiv.org/abs/2607.11214v1
• Date: Mon, 13 Jul 2026 08:07:45 GMT
• Title: A Novel Method to Evaluate Models on Unreliable, Noisy and Inconsistent Labels: Adaptive Resolution Label Aggregation (ARLA)
• Abstract summary: "Adaptive Resolution Label Aggregation", or "ARLA", adapts the resolution of both the label and the model prediction at inference time.<n>We demonstrate how ARLA can be used to better analyse model behaviour with a practical application to a real flood prediction model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Labels are critical for both training and evaluating deep learning segmentation models, but are often inconsistent, noisy, or ambiguous at class boundaries. Many approaches have been developed to support training models on weak labels, but few to none currently exist to facilitate evaluating models on unreliable labels. We therefore introduce a method called "Adaptive Resolution Label Aggregation", or "ARLA", which dynamically adapts the resolution of both the label and the model prediction at inference time before the evaluation metrics are computed. We demonstrate how ARLA can be used to better analyse model behaviour with a practical application to a real flood prediction model, where ARLA was able to overcome issues with inconsistent labelling of forested areas and errors in labels within regions of heavy cloud cover. Our work presents a new approach to evaluating segmentation models, with adjustable parameters to adapt the aggregated resolution to the precision of the label or the level of label noise. Fundamentally, ARLA exploits the information encapsulated by a label but minimises the label error, extracting from the noise a clearer signal of a model's true performance.

Related papers

• PaSta: Noisy Node Classification with Partial Label Learning [58.43127707101144]
  We propose a novel Partial label-based Self-training framework to overcome the limitations of existing methods.<n>PaSta achieves an average improvement of 1.1% in classification performance under various noise settings.
  arXiv  Detail & Related papers  (2026-08-26T04:40:23Z)
• Sharpness-aware Dynamic Anchor Selection for Generalized Category Discovery [61.694524826522205]
  Given some labeled data of known classes, GCD aims to cluster unlabeled data that contain both known and unknown classes.<n>Large pre-trained models have a preference for some specific visual patterns, resulting in encoding spurious correlation for unlabeled data.<n>We propose a novel method, which contains two modules: Loss Sharpness Penalty (LSP) and Dynamic Anchor Selection (DAS)
  arXiv  Detail & Related papers  (2025-12-15T02:24:06Z)
• Ordinal Adaptive Correction: A Data-Centric Approach to Ordinal Image Classification with Noisy Labels [0.9023847175654603]
  ORDinal Adaptive Correction (ORDAC) is proposed for adaptive correction of noisy labels.<n>During training, ORDAC dynamically adjusts the mean and standard deviation of the label distribution for each sample.<n>Results show that ORDAC and its extended versions lead to significant improvements in model performance.
  arXiv  Detail & Related papers  (2025-09-02T14:17:16Z)
• Efficient Adaptive Label Refinement for Label Noise Learning [14.617885790129336]
  We propose Adaptive Label Refinement (ALR) to avoid incorrect labels and thoroughly learning clean samples.<n>ALR is simple and efficient, requiring no prior knowledge of noise or auxiliary datasets.<n>We validate ALR's effectiveness through experiments on benchmark datasets with artificial label noise (CIFAR-10/100) and real-world datasets with inherent noise (ANIMAL-10N, Clothing1M, WebVision)
  arXiv  Detail & Related papers  (2025-02-01T09:58:08Z)
• Unsupervised Domain Adaptation for Semantic Segmentation with Pseudo Label Self-Refinement [9.69089112870202]
  We propose an auxiliary pseudo-label refinement network (PRN) for online refining of the pseudo labels and also localizing the pixels whose predicted labels are likely to be noisy. We evaluate our approach on benchmark datasets with three different domain shifts, and our approach consistently performs significantly better than the previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-10-25T20:31:07Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• Category-Adaptive Label Discovery and Noise Rejection for Multi-label Image Recognition with Partial Positive Labels [78.88007892742438]
  Training multi-label models with partial positive labels (MLR-PPL) attracts increasing attention. Previous works regard unknown labels as negative and adopt traditional MLR algorithms. We propose to explore semantic correlation among different images to facilitate the MLR-PPL task.
  arXiv  Detail & Related papers  (2022-11-15T02:11:20Z)
• Transductive CLIP with Class-Conditional Contrastive Learning [68.51078382124331]
  We propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. A class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels. ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels.
  arXiv  Detail & Related papers  (2022-06-13T14:04:57Z)
• Progressive Identification of True Labels for Partial-Label Learning [112.94467491335611]
  Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label. Most existing methods elaborately designed as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data. This paper proposes a novel framework of classifier with flexibility on the model and optimization algorithm.
  arXiv  Detail & Related papers  (2020-02-19T08:35:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.