Better than Average: Spatially-Aware Aggregation of Segmentation Uncertainty Improves Downstream Performance

• URL: http://arxiv.org/abs/2603.29941v1
• Date: Tue, 31 Mar 2026 16:15:26 GMT
• Title: Better than Average: Spatially-Aware Aggregation of Segmentation Uncertainty Improves Downstream Performance
• Abstract summary: Uncertainty Quantification (UQ) is crucial for ensuring the reliability of automated image segmentations in safety-critical domains like biomedical image analysis or autonomous driving.<n>In segmentation, UQ generates pixel-wise uncertainty scores that must be aggregated into image-level scores for downstream tasks like Out-of-Distribution (OoD) or failure detection.<n>Despite routine use of aggregation strategies, their properties and impact on downstream task performance have not yet been comprehensively studied.
• Score: 5.331671515360591
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uncertainty Quantification (UQ) is crucial for ensuring the reliability of automated image segmentations in safety-critical domains like biomedical image analysis or autonomous driving. In segmentation, UQ generates pixel-wise uncertainty scores that must be aggregated into image-level scores for downstream tasks like Out-of-Distribution (OoD) or failure detection. Despite routine use of aggregation strategies, their properties and impact on downstream task performance have not yet been comprehensively studied. Global Average is the default choice, yet it does not account for spatial and structural features of segmentation uncertainty. Alternatives like patch-, class- and threshold-based strategies exist, but lack systematic comparison, leading to inconsistent reporting and unclear best practices. We address this gap by (1) formally analyzing properties, limitations, and pitfalls of common strategies; (2) proposing novel strategies that incorporate spatial uncertainty structure and (3) benchmarking their performance on OoD and failure detection across ten datasets that vary in image geometry and structure. We find that aggregators leveraging spatial structure yield stronger performance in both downstream tasks studied. However, the performance of individual aggregators depends heavily on dataset characteristics, so we (4) propose a meta-aggregator that integrates multiple aggregators and performs robustly across datasets.

Related papers

• Label-Free Target-Domain Adaptation for Unconstrained Event-Image Feature Matching via Dual-Stage Distillation [30.789763956644958]
  Cross-modal matching methods are largely restricted by their reliance on either matching labels or strictly aligned hardware.<n>We propose a novel two-stage training paradigm to address this problem.<n>Our proposed method achieves state-of-the-art performance on both MVSEC and TUM-VIE pose estimation tasks.
  arXiv  Detail & Related papers  (2026-07-11T02:23:11Z)
• Unsupervised Domain Adaptation for Sim-to-Real Object Pose Estimation with Contrastive Alignment and Pseudo-Label Refinement [14.854437007037683]
  Unsupervised domain adaptation (UDA) enables robust transfer of knowledge from simulated to real environments.<n>We introduce CAPLR that targets the adaptation of pose-sensitive features in localized regions.<n>We show that CAPLR achieves state-of-the-art performance across multiple well-known object pose estimation benchmarks.
  arXiv  Detail & Related papers  (2026-06-19T10:11:17Z)
• A Systematic Approach for Selecting Trajectories for Data Augmentation [0.0]
  This thesis develops a framework to evaluate five systematic selection strategies: Outlierness, Diversity, Representativeness, Uncertainty, and Random selection.<n>These strategies were rigorously tested across four datasets covering animal behavior, maritime traffic, and urban traffic.<n>The results indicate that, while systematic selection is not a universal solution, it offers advantages over the random baseline.
  arXiv  Detail & Related papers  (2026-06-09T14:46:47Z)
• A Hierarchical Error-Corrective Graph Framework for Autonomous Agents with LLM-Based Action Generation [3.503249117641051]
  We propose a Hierarchical Error-Corrective Graph FrameworkforAutonomousAgentswithLLM-BasedActionGeneration(HECG),whichincorporates three core innovations.<n>MDTS: by integrating task quality metrics (Q), confidence/cost metrics (C), reward metrics (R), and LLM-based semantic reasoning scores (LLM-Score), MDTS achieves multi-dimensional alignment between quantitative performance and semantic context.<n> EMC: unlike simple confusion matrices or overall performance metrics, EMC provides structured attribution of task failures by categorizing errors into ten types, such as Strategy Errors (Strategy Whe) and Script
  arXiv  Detail & Related papers  (2026-03-09T13:46:00Z)
• Refining Decision Boundaries In Anomaly Detection Using Similarity Search Within the Feature Space [3.3202103799131795]
  We introduce SDA2E, a Sparse Dual Adversarial Attention-based AutoEncoder designed to learn compact and discriminative latent representations from imbalanced, high-dimensional data.<n>We propose a similarity-guided active learning framework that integrates three novel strategies to refine decision boundaries efficiently.<n>We evaluate SDA2E extensively across 52 imbalanced datasets, including multiple DARPA Transparent Computing scenarios, and benchmark it against 15 state-of-the-art anomaly detection methods.
  arXiv  Detail & Related papers  (2026-02-02T23:55:08Z)
• Registration is a Powerful Rotation-Invariance Learner for 3D Anomaly Detection [64.0168648353038]
  3D anomaly detection in point-cloud data is critical for industrial quality control, aiming to identify structural defects with high reliability.<n>Current memory bank-based methods often suffer from inconsistent feature transformations and limited discriminative capacity.<n>We propose a registration-induced, rotation-invariant feature extraction framework that integrates the objectives of point-cloud registration and memory-based anomaly detection.
  arXiv  Detail & Related papers  (2025-10-19T14:56:38Z)
• Explainable Human-in-the-Loop Segmentation via Critic Feedback Signals [0.20999222360659608]
  We propose a human-in-the-loop interactive framework that enables interventional learning through targeted human corrections of segmentation outputs.<n>We demonstrate that our framework improves segmentation accuracy by up to 9 mIoU points on challenging cubemap data.<n>This work provides a practical framework for researchers and practitioners seeking to build segmentation systems that are accurate, robust to dataset biases, data-efficient, and adaptable to real-world domains such as urban climate monitoring and autonomous driving.
  arXiv  Detail & Related papers  (2025-10-11T01:16:41Z)
• Leveraging Ensembles and Self-Supervised Learning for Fully-Unsupervised Person Re-Identification and Text Authorship Attribution [77.85461690214551]
  Learning from fully-unlabeled data is challenging in Multimedia Forensics problems, such as Person Re-Identification and Text Authorship Attribution. Recent self-supervised learning methods have shown to be effective when dealing with fully-unlabeled data in cases where the underlying classes have significant semantic differences. We propose a strategy to tackle Person Re-Identification and Text Authorship Attribution by enabling learning from unlabeled data even when samples from different classes are not prominently diverse.
  arXiv  Detail & Related papers  (2022-02-07T13:08:11Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Unsupervised Robust Domain Adaptation without Source Data [75.85602424699447]
  We study the problem of robust domain adaptation in the context of unavailable target labels and source data. We show a consistent performance improvement of over $10%$ in accuracy against the tested baselines on four benchmark datasets.
  arXiv  Detail & Related papers  (2021-03-26T16:42:28Z)
• Unsupervised and self-adaptative techniques for cross-domain person re-identification [82.54691433502335]
  Person Re-Identification (ReID) across non-overlapping cameras is a challenging task. Unsupervised Domain Adaptation (UDA) is a promising alternative, as it performs feature-learning adaptation from a model trained on a source to a target domain without identity-label annotation. In this paper, we propose a novel UDA-based ReID method that takes advantage of triplets of samples created by a new offline strategy.
  arXiv  Detail & Related papers  (2021-03-21T23:58:39Z)
• Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning [85.6386289476598]
  We develop a novel adversarial graph representation adaptation (AGRA) framework for cross-domain holistic-local feature co-adaptation. We conduct extensive and fair evaluations on several popular benchmarks and show that the proposed AGRA framework outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-08-03T15:00:31Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.