Med-SegLens: Latent-Level Model Diffing for Interpretable Medical Image Segmentation

• URL: http://arxiv.org/abs/2602.10508v1
• Date: Wed, 11 Feb 2026 04:15:36 GMT
• Title: Med-SegLens: Latent-Level Model Diffing for Interpretable Medical Image Segmentation
• Authors: Salma J. Ahmed, Emad A. Mohammed, Azam Asilian Bidgoli,
• Abstract summary: We introduce Med-SegLens, a model-diffing framework that decomposes segmentation model activations into interpretable latent features.<n>We show that latent-level interventions can correct errors and improve cross-dataset adaption without retraining.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern segmentation models achieve strong predictive performance but remain largely opaque, limiting our ability to diagnose failures, understand dataset shift, or intervene in a principled manner. We introduce Med-SegLens, a model-diffing framework that decomposes segmentation model activations into interpretable latent features using sparse autoencoders trained on SegFormer and U-Net. Through cross-architecture and cross-dataset latent alignment across healthy, adult, pediatric, and sub-Saharan African glioma cohorts, we identify a stable backbone of shared representations, while dataset shift is driven by differential reliance on population-specific latents. We show that these latents act as causal bottlenecks for segmentation failures, and that targeted latent-level interventions can correct errors and improve cross-dataset adaption without retraining, recovering performance in 70% of failure cases and improving Dice score from 39.4% to 74.2%. Our results demonstrate that latent-level model diffing provides a practical and mechanistic tool for diagnosing failures and mitigating dataset shift in segmentation models.

Related papers

• The Mean is the Mirage: Entropy-Adaptive Model Merging under Heterogeneous Domain Shifts in Medical Imaging [3.597779662054083]
  Model merging under unseen test-time distribution shifts often renders naive strategies, such as mean averaging unreliable.<n>We introduce an entropy-adaptive, fully online model-merging method that yields a batch-specific merged model via only forward passes.<n>We extensively evaluate our method with state-of-the-art baselines using two backbones across nine medical and natural-domain generalization image classification datasets.
  arXiv  Detail & Related papers  (2026-02-24T21:06:19Z)
• Automated Lesion Segmentation of Stroke MRI Using nnU-Net: A Comprehensive External Validation Across Acute and Chronic Lesions [0.0]
  We evaluate stroke lesion segmentation using the nnU-Net framework across multiple publicly available MRI datasets.<n>Across stroke stages, models showed robust generalisation, with segmentation accuracy approaching reported inter-rater reliability.<n>In acute stroke, DWI-trained models consistently outperformed FLAIR-based models, with only modest gains from multimodal combinations.<n>In chronic stroke, increasing training set size improved performance, with diminishing returns beyond several hundred cases.
  arXiv  Detail & Related papers  (2026-01-13T16:29:20Z)
• UniSegDiff: Boosting Unified Lesion Segmentation via a Staged Diffusion Model [53.34835793648352]
  We propose UniSegDiff, a novel diffusion model framework for lesion segmentation.<n>UniSegDiff addresses lesion segmentation in a unified manner across multiple modalities and organs.<n> Comprehensive experimental results demonstrate that UniSegDiff significantly outperforms previous state-of-the-art (SOTA) approaches.
  arXiv  Detail & Related papers  (2025-07-24T12:33:10Z)
• Differential-UMamba: Rethinking Tumor Segmentation Under Limited Data Scenarios [3.1231963031043786]
  We introduce Diff-UMamba, a novel architecture that combines the UNet framework with the mamba mechanism to model long-range dependencies.<n>At the heart of Diff-UMamba is a noise reduction module, which employs a signal differencing strategy to suppress noisy or irrelevant activations.<n>The architecture achieves improved segmentation accuracy and robustness, particularly in low-data settings.
  arXiv  Detail & Related papers  (2025-07-24T08:23:11Z)
• Uncertainty-Error correlations in Evidential Deep Learning models for biomedical segmentation [0.0]
  Evidential Deep Learning is applied in the context of biomedical image segmentation. We found that Evidential Deep Learning models with U-Net backbones generally yielded superior correlations between prediction errors and uncertainties. These superior features of EDL models render them well-suited for segmentation tasks that warrant a critical sensitivity in detecting large model errors.
  arXiv  Detail & Related papers  (2024-10-24T06:16:04Z)
• Optimizing against Infeasible Inclusions from Data for Semantic Segmentation through Morphology [58.17907376475596]
  State-of-the-art semantic segmentation models are typically optimized in a data-driven fashion.<n>InSeIn extracts explicit inclusion constraints that govern spatial class relations from the semantic segmentation training set at hand.<n>It then enforces a morphological yet differentiable loss that penalizes violations of these constraints during training to promote prediction feasibility.
  arXiv  Detail & Related papers  (2024-08-26T22:39:08Z)
• Automated Detection of Myopic Maculopathy in MMAC 2023: Achievements in Classification, Segmentation, and Spherical Equivalent Prediction [6.993091116816899]
  Myopic macular degeneration is the primary cause of vision loss in individuals with pathological myopia. Early detection and prompt treatment are crucial in preventing vision impairment due to myopic maculopathy. This was the focus of the Myopic Maculopathy Analysis Challenge (MMAC)
  arXiv  Detail & Related papers  (2024-01-08T00:26:21Z)
• PULASki: Learning inter-rater variability using statistical distances to improve probabilistic segmentation [35.34932609930401]
  This work proposes the PULASki method as a computationally efficient generative tool for biomedical image segmentation.<n>It captures variability in expert annotations, even in small datasets.<n>Our experiments are also the first to present a comparative study of the computationally feasible segmentation of complex geometries using 3D patches and the traditional use of 2D slices.
  arXiv  Detail & Related papers  (2023-12-25T10:31:22Z)
• Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
  We propose an ensemble modeling approach for multimodal action recognition. We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
  arXiv  Detail & Related papers  (2023-08-10T08:43:20Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Towards Reliable Medical Image Segmentation by Modeling Evidential Calibrated Uncertainty [57.023423137202485]
  Concerns regarding the reliability of medical image segmentation persist among clinicians.<n>We introduce DEviS, an easily implementable foundational model that seamlessly integrates into various medical image segmentation networks.<n>By leveraging subjective logic theory, we explicitly model probability and uncertainty for medical image segmentation.
  arXiv  Detail & Related papers  (2023-01-01T05:02:46Z)
• Towards Robust Partially Supervised Multi-Structure Medical Image Segmentation on Small-Scale Data [123.03252888189546]
  We propose Vicinal Labels Under Uncertainty (VLUU) to bridge the methodological gaps in partially supervised learning (PSL) under data scarcity. Motivated by multi-task learning and vicinal risk minimization, VLUU transforms the partially supervised problem into a fully supervised problem by generating vicinal labels. Our research suggests a new research direction in label-efficient deep learning with partial supervision.
  arXiv  Detail & Related papers  (2020-11-28T16:31:00Z)
• Diversity inducing Information Bottleneck in Model Ensembles [73.80615604822435]
  In this paper, we target the problem of generating effective ensembles of neural networks by encouraging diversity in prediction. We explicitly optimize a diversity inducing adversarial loss for learning latent variables and thereby obtain diversity in the output predictions necessary for modeling multi-modal data. Compared to the most competitive baselines, we show significant improvements in classification accuracy, under a shift in the data distribution.
  arXiv  Detail & Related papers  (2020-03-10T03:10:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.