MiSuRe is all you need to explain your image segmentation

• URL: http://arxiv.org/abs/2406.12173v2
• Date: Wed, 19 Jun 2024 00:27:37 GMT
• Title: MiSuRe is all you need to explain your image segmentation
• Authors: Syed Nouman Hasany, Fabrice Mériaudeau, Caroline Petitjean,
• Abstract summary: We propose MiSuRe as an algorithm to generate saliency maps for image segmentation. The goal of the saliency maps generated by MiSuRe is to get rid of irrelevant regions, and only highlight those regions in the input image which are crucial to the image segmentation decision.
• Score: 3.363736106489207
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The last decade of computer vision has been dominated by Deep Learning architectures, thanks to their unparalleled success. Their performance, however, often comes at the cost of explainability owing to their highly non-linear nature. Consequently, a parallel field of eXplainable Artificial Intelligence (XAI) has developed with the aim of generating insights regarding the decision making process of deep learning models. An important problem in XAI is that of the generation of saliency maps. These are regions in an input image which contributed most towards the model's final decision. Most work in this regard, however, has been focused on image classification, and image segmentation - despite being a ubiquitous task - has not received the same attention. In the present work, we propose MiSuRe (Minimally Sufficient Region) as an algorithm to generate saliency maps for image segmentation. The goal of the saliency maps generated by MiSuRe is to get rid of irrelevant regions, and only highlight those regions in the input image which are crucial to the image segmentation decision. We perform our analysis on 3 datasets: Triangle (artificially constructed), COCO-2017 (natural images), and the Synapse multi-organ (medical images). Additionally, we identify a potential usecase of these post-hoc saliency maps in order to perform post-hoc reliability of the segmentation model.

Related papers

• Lidar Annotation Is All You Need [0.0]
  This paper aims to improve the efficiency of image segmentation using a convolutional neural network in a multi-sensor setup. The key innovation of our approach is the masked loss, addressing sparse ground-truth masks from point clouds. Experimental validation of the approach on benchmark datasets shows comparable performance to a high-quality image segmentation model.
  arXiv  Detail & Related papers  (2023-11-08T15:55:18Z)
• R-MAE: Regions Meet Masked Autoencoders [113.73147144125385]
  We explore regions as a potential visual analogue of words for self-supervised image representation learning. Inspired by Masked Autoencoding (MAE), a generative pre-training baseline, we propose masked region autoencoding to learn from groups of pixels or regions.
  arXiv  Detail & Related papers  (2023-06-08T17:56:46Z)
• A Shared Representation for Photorealistic Driving Simulators [83.5985178314263]
  We propose to improve the quality of generated images by rethinking the discriminator architecture. The focus is on the class of problems where images are generated given semantic inputs, such as scene segmentation maps or human body poses. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning.
  arXiv  Detail & Related papers  (2021-12-09T18:59:21Z)
• Unsupervised Part Discovery from Contrastive Reconstruction [90.88501867321573]
  The goal of self-supervised visual representation learning is to learn strong, transferable image representations. We propose an unsupervised approach to object part discovery and segmentation. Our method yields semantic parts consistent across fine-grained but visually distinct categories.
  arXiv  Detail & Related papers  (2021-11-11T17:59:42Z)
• BI-GCN: Boundary-Aware Input-Dependent Graph Convolution Network for Biomedical Image Segmentation [21.912509900254364]
  We apply graph convolution into the segmentation task and propose an improved textitLaplacian. Our method outperforms the state-of-the-art approaches on the segmentation of polyps in colonoscopy images and of the optic disc and optic cup in colour fundus images.
  arXiv  Detail & Related papers  (2021-10-27T21:12:27Z)
• CAMERAS: Enhanced Resolution And Sanity preserving Class Activation Mapping for image saliency [61.40511574314069]
  Backpropagation image saliency aims at explaining model predictions by estimating model-centric importance of individual pixels in the input. We propose CAMERAS, a technique to compute high-fidelity backpropagation saliency maps without requiring any external priors.
  arXiv  Detail & Related papers  (2021-06-20T08:20:56Z)
• PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation [87.50205728818601]
  We propose a PriorGuided Local (PGL) self-supervised model that learns the region-wise local consistency in the latent feature space. Our PGL model learns the distinctive representations of local regions, and hence is able to retain structural information.
  arXiv  Detail & Related papers  (2020-11-25T11:03:11Z)
• Semantic Editing On Segmentation Map Via Multi-Expansion Loss [98.1131339357174]
  This paper aims to improve quality of edited segmentation map conditioned on semantic inputs. We propose MExGAN for semantic editing on segmentation map, which uses a novel Multi-Expansion (MEx) loss. Experiments on semantic editing on segmentation map and natural image inpainting show competitive results on four datasets.
  arXiv  Detail & Related papers  (2020-10-16T03:12:26Z)
• AinnoSeg: Panoramic Segmentation with High Perfomance [4.867465475957119]
  Current panoramic segmentation algorithms are more concerned with context semantics, but the details of image are not processed enough. Aiming to address these issues, this paper presents some useful tricks. All these operations named AinnoSeg, AinnoSeg can achieve state-of-art performance on the well-known dataset ADE20K.
  arXiv  Detail & Related papers  (2020-07-21T04:16:46Z)
• Manifold-driven Attention Maps for Weakly Supervised Segmentation [9.289524646688244]
  We propose a manifold driven attention-based network to enhance visual salient regions. Our method generates superior attention maps directly during inference without the need of extra computations.
  arXiv  Detail & Related papers  (2020-04-07T00:03:28Z)
• Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey [0.0]
  We take a glance at the evolution of both semantic and instance segmentation work based on CNN. We have given a glimpse of some state-of-the-art panoptic segmentation models.
  arXiv  Detail & Related papers  (2020-01-13T06:07:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.