Aggregated Attributions for Explanatory Analysis of 3D Segmentation Models

• URL: http://arxiv.org/abs/2407.16653v3
• Date: Fri, 29 Nov 2024 14:15:43 GMT
• Title: Aggregated Attributions for Explanatory Analysis of 3D Segmentation Models
• Authors: Maciej Chrabaszcz, Hubert Baniecki, Piotr Komorowski, Szymon Płotka, Przemyslaw Biecek,
• Abstract summary: We introduce Agg2Exp, a methodology for aggregating fine-grained voxel attributions of 3D segmentation models.<n>Our experiments show that gradient-based voxel attributions are more faithful to the model's predictions than perturbation-based explanations.<n>Agg2Exp facilitates the explanatory analysis of large segmentation models beyond their predictive performance.
• Score: 7.416913210816592
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Analysis of 3D segmentation models, especially in the context of medical imaging, is often limited to segmentation performance metrics that overlook the crucial aspect of explainability and bias. Currently, effectively explaining these models with saliency maps is challenging due to the high dimensions of input images multiplied by the ever-growing number of segmented class labels. To this end, we introduce Agg^2Exp, a methodology for aggregating fine-grained voxel attributions of the segmentation model's predictions. Unlike classical explanation methods that primarily focus on the local feature attribution, Agg^2Exp enables a more comprehensive global view on the importance of predicted segments in 3D images. Our benchmarking experiments show that gradient-based voxel attributions are more faithful to the model's predictions than perturbation-based explanations. As a concrete use-case, we apply Agg^2Exp to discover knowledge acquired by the Swin UNEt TRansformer model trained on the TotalSegmentator v2 dataset for segmenting anatomical structures in computed tomography medical images. Agg^2Exp facilitates the explanatory analysis of large segmentation models beyond their predictive performance. The source code is publicly available at https://github.com/mi2datalab/agg2exp.

Related papers

• MedNeXt-v2: Scaling 3D ConvNeXts for Large-Scale Supervised Representation Learning in Medical Image Segmentation [6.241063761137199]
  Large-scale supervised pretraining is rapidly reshaping 3D medical image segmentation.<n>We show that routinely used backbones in large-scale pretraining pipelines are often suboptimal.<n>We introduce MedNeXt-v2, a compound-scaled 3D ConvNeXt that leverages improved micro-architecture and data scaling to deliver state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-19T16:45:23Z)
• In search of truth: Evaluating concordance of AI-based anatomy segmentation models [3.740726797046942]
  AI-based methods for anatomy segmentation can help automate characterization of large imaging datasets.<n>We introduce a practical framework to assist in evaluating them on datasets that do not contain ground truth annotations.
  arXiv  Detail & Related papers  (2025-12-17T19:33:56Z)
• OpenInsGaussian: Open-vocabulary Instance Gaussian Segmentation with Context-aware Cross-view Fusion [89.98812408058336]
  We introduce textbfOpenInsGaussian, an textbfOpen-vocabulary textbfInstance textbfGaussian segmentation framework with Context-aware Cross-view Fusion.<n>OpenInsGaussian achieves state-of-the-art results in open-vocabulary 3D Gaussian segmentation, outperforming existing baselines by a large margin.
  arXiv  Detail & Related papers  (2025-10-21T03:24:12Z)
• LangDAug: Langevin Data Augmentation for Multi-Source Domain Generalization in Medical Image Segmentation [5.9862846364925115]
  Domain Generalization (DG) methods overcome this either through representation learning or data augmentation (DAug)<n>We propose LangDAug, a novel $textbf$evin $textbfD$ata $textbfAug$mentation for multi-source domain generalization in 2D medical image segmentation.<n>We show that LangDAug outperforms state-of-the-art domain generalization methods and effectively complements existing domain-randomization approaches.
  arXiv  Detail & Related papers  (2025-05-26T08:18:32Z)
• A Graph-Based Framework for Interpretable Whole Slide Image Analysis [86.37618055724441]
  We develop a framework that transforms whole-slide images into biologically-informed graph representations.<n>Our approach builds graph nodes from tissue regions that respect natural structures, not arbitrary grids.<n>We demonstrate strong performance on challenging cancer staging and survival prediction tasks.
  arXiv  Detail & Related papers  (2025-03-14T20:15:04Z)
• GraphCL: Graph-based Clustering for Semi-Supervised Medical Image Segmentation [20.43162035511489]
  We propose a graph-based clustering for semi-supervised medical image segmentation (GraphCL) by jointly modeling graph data structure in a unified deep model. We show that the proposed GraphCL algorithm outperforms state-of-the-art semi-supervised medical image segmentation methods.
  arXiv  Detail & Related papers  (2024-11-20T09:24:46Z)
• ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining [104.34751911174196]
  We build a large-scale dataset of 3DGS using ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories. We introduce textbftextitGaussian-MAE, which highlights the unique benefits of representation learning from Gaussian parameters.
  arXiv  Detail & Related papers  (2024-08-20T14:49:14Z)
• MOUNTAINEER: Topology-Driven Visual Analytics for Comparing Local Explanations [6.835413642522898]
  Topological Data Analysis (TDA) can be an effective method in this domain since it can be used to transform attributions into uniform graph representations. We present a novel topology-driven visual analytics tool, Mountaineer, that allows ML practitioners to interactively analyze and compare these representations. We show how Mountaineer enabled us to compare black-box ML explanations and discern regions of and causes of disagreements between different explanations.
  arXiv  Detail & Related papers  (2024-06-21T19:28:50Z)
• View-Consistent Hierarchical 3D Segmentation Using Ultrametric Feature Fields [52.08335264414515]
  We learn a novel feature field within a Neural Radiance Field (NeRF) representing a 3D scene. Our method takes view-inconsistent multi-granularity 2D segmentations as input and produces a hierarchy of 3D-consistent segmentations as output. We evaluate our method and several baselines on synthetic datasets with multi-view images and multi-granular segmentation, showcasing improved accuracy and viewpoint-consistency.
  arXiv  Detail & Related papers  (2024-05-30T04:14:58Z)
• Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models [2.355460994057843]
  LIME is a popular XAI framework for unraveling decision-making processes in vision machine-learning models. We introduce the DSEG-LIME (Data-Driven LIME) framework, featuring a data-driven segmentation for human-recognized feature generation. Our findings demonstrate that DSEG outperforms on several XAI metrics on pre-trained ImageNet models.
  arXiv  Detail & Related papers  (2024-03-12T15:13:12Z)
• Prospector Heads: Generalized Feature Attribution for Large Models & Data [82.02696069543454]
  We introduce prospector heads, an efficient and interpretable alternative to explanation-based attribution methods. We demonstrate how prospector heads enable improved interpretation and discovery of class-specific patterns in input data.
  arXiv  Detail & Related papers  (2024-02-18T23:01:28Z)
• Topological Data Analysis Guided Segment Anything Model Prompt Optimization for Zero-Shot Segmentation in Biological Imaging [5.795215830149858]
  We propose topological data analysis guided prompt optimization for the Segment Anything Model (SAM) Our results show that the TDA optimized point cloud is much better suited for finding small objects and massively reduces computational complexity.
  arXiv  Detail & Related papers  (2023-06-30T05:00:38Z)
• Learning with Explicit Shape Priors for Medical Image Segmentation [17.110893665132423]
  We propose a novel shape prior module (SPM) to promote the segmentation performance of UNet-based models. Explicit shape priors consist of global and local shape priors. Our proposed model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-03-31T11:12:35Z)
• Semi-Weakly Supervised Object Kinematic Motion Prediction [56.282759127180306]
  Given a 3D object, kinematic motion prediction aims to identify the mobile parts as well as the corresponding motion parameters. We propose a graph neural network to learn the map between hierarchical part-level segmentation and mobile parts parameters. The network predictions yield a large scale of 3D objects with pseudo labeled mobility information.
  arXiv  Detail & Related papers  (2023-03-31T02:37:36Z)
• SegPrompt: Using Segmentation Map as a Better Prompt to Finetune Deep Models for Kidney Stone Classification [62.403510793388705]
  Deep learning has produced encouraging results for kidney stone classification using endoscope images. The shortage of annotated training data poses a severe problem in improving the performance and generalization ability of the trained model. We propose SegPrompt to alleviate the data shortage problems by exploiting segmentation maps from two aspects.
  arXiv  Detail & Related papers  (2023-03-15T01:30:48Z)
• Unsupervised Domain Adaptation through Shape Modeling for Medical Image Segmentation [23.045760366698634]
  We aim at modeling shape explicitly and using it to help medical image segmentation. Previous methods proposed Variational Autoencoder (VAE) based models to learn the distribution of shape for a particular organ. We propose a new unsupervised domain adaptation pipeline based on a pseudo loss and a VAE reconstruction loss under a teacher-student learning paradigm.
  arXiv  Detail & Related papers  (2022-07-06T09:16:42Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• Class-wise Dynamic Graph Convolution for Semantic Segmentation [63.08061813253613]
  We propose a class-wise dynamic graph convolution (CDGC) module to adaptively propagate information. We also introduce the Class-wise Dynamic Graph Convolution Network(CDGCNet), which consists of two main parts including the CDGC module and a basic segmentation network. We conduct extensive experiments on three popular semantic segmentation benchmarks including Cityscapes, PASCAL VOC 2012 and COCO Stuff.
  arXiv  Detail & Related papers  (2020-07-19T15:26:50Z)
• Monocular Human Pose and Shape Reconstruction using Part Differentiable Rendering [53.16864661460889]
  Recent works succeed in regression-based methods which estimate parametric models directly through a deep neural network supervised by 3D ground truth. In this paper, we introduce body segmentation as critical supervision. To improve the reconstruction with part segmentation, we propose a part-level differentiable part that enables part-based models to be supervised by part segmentation.
  arXiv  Detail & Related papers  (2020-03-24T14:25:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.