Shape-Aware Masking for Inpainting in Medical Imaging

• URL: http://arxiv.org/abs/2207.05787v1
• Date: Tue, 12 Jul 2022 18:35:17 GMT
• Title: Shape-Aware Masking for Inpainting in Medical Imaging
• Authors: Yousef Yeganeh, Azade Farshad, Nassir Navab
• Abstract summary: Inpainting has been proposed as a successful deep learning technique for unsupervised medical image model discovery. We introduce a method for generating shape-aware masks for inpainting, which aims at learning the statistical shape prior. We propose an unsupervised guided masking approach based on an off-the-shelf inpainting model and a superpixel over-segmentation algorithm.
• Score: 49.61617087640379
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inpainting has recently been proposed as a successful deep learning technique for unsupervised medical image model discovery. The masks used for inpainting are generally independent of the dataset and are not tailored to perform on different given classes of anatomy. In this work, we introduce a method for generating shape-aware masks for inpainting, which aims at learning the statistical shape prior. We hypothesize that although the variation of masks improves the generalizability of inpainting models, the shape of the masks should follow the topology of the organs of interest. Hence, we propose an unsupervised guided masking approach based on an off-the-shelf inpainting model and a superpixel over-segmentation algorithm to generate a wide range of shape-dependent masks. Experimental results on abdominal MR image reconstruction show the superiority of our proposed masking method over standard methods using square-shaped or dataset of irregular shape masks.

Related papers

• Inpainting-Driven Mask Optimization for Object Removal [15.429649454099085]
  This paper proposes a mask optimization method for improving the quality of object removal using image inpainting. In our method, this domain gap is resolved by training the inpainting network with object masks extracted by segmentation. To optimize the object masks for inpainting, the segmentation network is connected to the inpainting network and end-to-end trained to improve the inpainting performance.
  arXiv  Detail & Related papers  (2024-03-23T13:52:16Z)
• Unsupervised Anomaly Detection in Medical Images Using Masked Diffusion Model [7.116982044576858]
  Masked Image Modeling (MIM) and Masked Frequency Modeling (MFM) in our self-supervised approach that enables models to learn visual representations from unlabeled data. We evaluate our approach on datasets containing tumors and numerous sclerosis lesions.
  arXiv  Detail & Related papers  (2023-05-31T14:04:11Z)
• PaintSeg: Training-free Segmentation via Painting [50.17936803209125]
  PaintSeg is a new unsupervised method for segmenting objects without any training. Inpainting and outpainting are alternated, with the former masking the foreground and filling in the background, and the latter masking the background while recovering the missing part of the foreground object. Our experimental results demonstrate that PaintSeg outperforms existing approaches in coarse mask-prompt, box-prompt, and point-prompt segmentation tasks.
  arXiv  Detail & Related papers  (2023-05-30T20:43:42Z)
• Improving Masked Autoencoders by Learning Where to Mask [65.89510231743692]
  Masked image modeling is a promising self-supervised learning method for visual data. We present AutoMAE, a framework that uses Gumbel-Softmax to interlink an adversarially-trained mask generator and a mask-guided image modeling process. In our experiments, AutoMAE is shown to provide effective pretraining models on standard self-supervised benchmarks and downstream tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:28:55Z)
• MPS-AMS: Masked Patches Selection and Adaptive Masking Strategy Based Self-Supervised Medical Image Segmentation [46.76171191827165]
  We propose masked patches selection and adaptive masking strategy based self-supervised medical image segmentation method, named MPS-AMS. Our proposed method greatly outperforms the state-of-the-art self-supervised baselines.
  arXiv  Detail & Related papers  (2023-02-27T11:57:06Z)
• Towards Improved Input Masking for Convolutional Neural Networks [66.99060157800403]
  We propose a new masking method for CNNs we call layer masking. We show that our method is able to eliminate or minimize the influence of the mask shape or color on the output of the model. We also demonstrate how the shape of the mask may leak information about the class, thus affecting estimates of model reliance on class-relevant features.
  arXiv  Detail & Related papers  (2022-11-26T19:31:49Z)
• Layered Depth Refinement with Mask Guidance [61.10654666344419]
  We formulate a novel problem of mask-guided depth refinement that utilizes a generic mask to refine the depth prediction of SIDE models. Our framework performs layered refinement and inpainting/outpainting, decomposing the depth map into two separate layers signified by the mask and the inverse mask. We empirically show that our method is robust to different types of masks and initial depth predictions, accurately refining depth values in inner and outer mask boundary regions.
  arXiv  Detail & Related papers  (2022-06-07T06:42:44Z)
• RePaint: Inpainting using Denoising Diffusion Probabilistic Models [161.74792336127345]
  Free-form inpainting is the task of adding new content to an image in the regions specified by an arbitrary binary mask. We propose RePaint: A Denoising Probabilistic Model (DDPM) based inpainting approach that is applicable to even extreme masks. We validate our method for both faces and general-purpose image inpainting using standard and extreme masks.
  arXiv  Detail & Related papers  (2022-01-24T18:40:15Z)
• A 3D model-based approach for fitting masks to faces in the wild [9.958467179573235]
  We present a 3D model-based approach called WearMask3D for augmenting face images of various poses to the masked face counterparts. Our method proceeds by first fitting a 3D morphable model on the input image, second overlaying the mask surface onto the face model and warping the respective mask texture, and last projecting the 3D mask back to 2D. Experimental results demonstrate WearMask3D produces more realistic masked images, and utilizing these images for training leads to improved recognition accuracy of masked faces.
  arXiv  Detail & Related papers  (2021-03-01T06:50:18Z)
• Iterative Facial Image Inpainting using Cyclic Reverse Generator [0.913755431537592]
  Cyclic Reverse Generator (CRG) architecture provides an encoder-generator model. We empirically observed that only a few iterations are sufficient to generate realistic images with the proposed model. Our method allows applying sketch-based inpaintings, using variety of mask types, and producing multiple and diverse results.
  arXiv  Detail & Related papers  (2021-01-18T12:19:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.