Reliability analysis for BraTS-GoAT segmentation: a controlled robustness study of deep-ensemble uncertainty

• URL: http://arxiv.org/abs/2608.13223v1
• Date: Thu, 13 Aug 2026 13:27:21 GMT
• Title: Reliability analysis for BraTS-GoAT segmentation: a controlled robustness study of deep-ensemble uncertainty
• Abstract summary: Deep networks segment brain tumours accurately in-distribution, but can fail silently when the input differs from their training data.<n>We ask how well a model segments, and whether its uncertainty knows when it is wrong.
• Score: 5.504955093712013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep networks segment brain tumours accurately in-distribution, but can fail silently when the input differs from their training data. That risk is central to clinical deployment and is the premise of the BraTS-GoAT generalizability task. We ask not only how well a model segments, but whether its uncertainty knows when it is wrong. On BraTS-GoAT (Task 3) we train a 5-fold cross-validated nnU-Net baseline (one held-out prediction per case) and a 3-seed deep ensemble. Both are evaluated for calibration and error detection on a per-region relevant mask, aggregated per case. In-distribution the 3-seed ensemble improves modestly over the already strong single model on the same held-out split, with the clearest gain in calibration. The separation appears under shift. In a controlled robustness study using graded synthetic corruptions as a proxy for acquisition shift, the single model's confidence stays flat while its accuracy and calibration degrade. Inter-member disagreement instead rises steeply, about a quarter to a third above the clean condition, several times the single model's response. On the official validation leaderboard the 5-fold ensemble of those folds attains whole-tumour Dice 0.87. The generalization gap is concentrated on the harder regions, with a characteristic failure of missing small, satellite lesions on unseen cohorts. In the synthetic study, disagreement among the 3-seed members is a more sensitive case-level indicator of acquisition shift than single-model confidence. Its per-voxel error localisation weakens as severity grows. The contribution is a rigorous, honest reliability comparison rather than a claim that any one uncertainty method dominates.

Related papers

• RePair: Turning Retrieval Failures into Counterfactual Hard Pairs [15.148342372814662]
  RePair mines false positives bidirectionally, applies LLM-guided counterfactual editing, and trains with a local hard-pair contrastive objective.<n>On Flickr30K and COCO30K, RePair outperforms controlled augmentation baselines with only 107K synthetic samples.
  arXiv  Detail & Related papers  (2026-08-30T06:47:38Z)
• The C-index illusion: discrimination without calibration in published survival models [0.0]
  We reproduce three published survival-ML models across three structurally distinct domains.<n>We test five pre-registered hypotheses under a Holm-corrected family-wise error rate.
  arXiv  Detail & Related papers  (2026-07-21T19:13:54Z)
• The Knowing-Saying Gap: When Probes See Errors that Confidence Misses [20.46211358172975]
  Linear probes detect corrupted context in language models with near-perfect accuracy, yet this does not translate into reliable failure prediction.<n>Across multi-hop arithmetic chains, probes that detect corruption turn out to be uninformative about final answer correctness.<n>As a real-time monitor, probe-based interventions are sharply model and error-type dependent.
  arXiv  Detail & Related papers  (2026-07-21T12:13:10Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• EnTrust: Modeling Inter-Modal Conflict for Trustworthy Multimodal Medical Image Analysis [41.67522319355107]
  EnTrust is a framework that treats inter-modal conflict as the primary source of predictive uncertainty.<n>It achieves state-of-the-art segmentation accuracy while reducing calibration error by 40% compared to the strongest baseline.
  arXiv  Detail & Related papers  (2026-06-19T12:43:50Z)
• Are We Overconfident in Models and Results for Semi-Supervised 3D Medical Image Segmentation? [4.120238673372104]
  Semi-supervised learning has become a dominant paradigm for reducing annotation costs.<n>We argue that the current progress is clouded by a twofold overconfidence problem.<n>We propose a tri-space principled segmentation framework founded on a dual-axis reliability assessment engine.
  arXiv  Detail & Related papers  (2026-05-25T08:16:40Z)
• Discovery of Hidden Miscalibration Regimes [52.452902154360565]
  Models may be systematically overconfident on some kinds of inputs and underconfident on others.<n>We define the corresponding miscalibration field and propose a diagnostic framework for estimating it.<n>Our approach learns a calibration-aware representation of the input space and estimates signed local miscalibration by kernel smoothing in the learned geometry.
  arXiv  Detail & Related papers  (2026-05-13T13:07:50Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• Benchmarking Corruption Robustness of LVLMs: A Discriminative Benchmark and Robustness Alignment Metric [49.393713730706445]
  We introduce Bench-C, a benchmark emphasizing discriminative samples for assessing corruption robustness.<n>We propose the Robustness Alignment Score (RAS), a unified metric that measures degradation in logit-level prediction structure.
  arXiv  Detail & Related papers  (2025-11-24T12:07:56Z)
• Parrot: Persuasion and Agreement Robustness Rating of Output Truth -- A Sycophancy Robustness Benchmark for LLMs [0.0]
  PARROT (Persuasion and Agreement Robustness Rating of Output Truth) is a robustness focused framework designed to measure the degradation in accuracy under social pressure exerted on users.<n>We evaluate 22 models using 1,302 MMLU-style multiple-choice questions across 13 domains and domain-specific authority templates.
  arXiv  Detail & Related papers  (2025-11-21T13:01:28Z)
• Beyond calibration: estimating the grouping loss of modern neural networks [68.8204255655161]
  Proper scoring rule theory shows that given the calibration loss, the missing piece to characterize individual errors is the grouping loss. We show that modern neural network architectures in vision and NLP exhibit grouping loss, notably in distribution shifts settings.
  arXiv  Detail & Related papers  (2022-10-28T07:04:20Z)
• Explicit Tradeoffs between Adversarial and Natural Distributional Robustness [48.44639585732391]
  In practice, models need to enjoy both types of robustness to ensure reliability. In this work, we show that in fact, explicit tradeoffs exist between adversarial and natural distributional robustness.
  arXiv  Detail & Related papers  (2022-09-15T19:58:01Z)
• Robustness and Accuracy Could Be Reconcilable by (Proper) Definition [109.62614226793833]
  The trade-off between robustness and accuracy has been widely studied in the adversarial literature. We find that it may stem from the improperly defined robust error, which imposes an inductive bias of local invariance. By definition, SCORE facilitates the reconciliation between robustness and accuracy, while still handling the worst-case uncertainty.
  arXiv  Detail & Related papers  (2022-02-21T10:36:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.