3D Matting: A Soft Segmentation Method Applied in Computed Tomography

• URL: http://arxiv.org/abs/2209.07843v1
• Date: Fri, 16 Sep 2022 10:18:59 GMT
• Title: 3D Matting: A Soft Segmentation Method Applied in Computed Tomography
• Authors: Lin Wang, Xiufen Ye, Donghao Zhang, Wanji He, Lie Ju, Xin Wang, Wei Feng, Kaimin Song, Xin Zhao, Zongyuan Ge
• Abstract summary: Three-dimensional (3D) images, such as CT, MRI, and PET, are common in medical imaging applications and important in clinical diagnosis. Semantic ambiguity is a typical feature of many medical image labels. In 2D medical images, using soft masks instead of binary masks generated by image matting to characterize lesions can provide rich semantic information.
• Score: 26.25446145993599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Three-dimensional (3D) images, such as CT, MRI, and PET, are common in medical imaging applications and important in clinical diagnosis. Semantic ambiguity is a typical feature of many medical image labels. It can be caused by many factors, such as the imaging properties, pathological anatomy, and the weak representation of the binary masks, which brings challenges to accurate 3D segmentation. In 2D medical images, using soft masks instead of binary masks generated by image matting to characterize lesions can provide rich semantic information, describe the structural characteristics of lesions more comprehensively, and thus benefit the subsequent diagnoses and analyses. In this work, we introduce image matting into the 3D scenes to describe the lesions in 3D medical images. The study of image matting in 3D modality is limited, and there is no high-quality annotated dataset related to 3D matting, therefore slowing down the development of data-driven deep-learning-based methods. To address this issue, we constructed the first 3D medical matting dataset and convincingly verified the validity of the dataset through quality control and downstream experiments in lung nodules classification. We then adapt the four selected state-of-the-art 2D image matting algorithms to 3D scenes and further customize the methods for CT images. Also, we propose the first end-to-end deep 3D matting network and implement a solid 3D medical image matting benchmark, which will be released to encourage further research.

Related papers

• E3D-GPT: Enhanced 3D Visual Foundation for Medical Vision-Language Model [23.56751925900571]
  The development of 3D medical vision-language models holds significant potential for disease diagnosis and patient treatment. We utilize self-supervised learning to construct a 3D visual foundation model for extracting 3D visual features. We apply 3D spatial convolutions to aggregate and project high-level image features, reducing computational complexity. Our model demonstrates superior performance compared to existing methods in report generation, visual question answering, and disease diagnosis.
  arXiv  Detail & Related papers  (2024-10-18T06:31:40Z)
• Autoregressive Sequence Modeling for 3D Medical Image Representation [48.706230961589924]
  We introduce a pioneering method for learning 3D medical image representations through an autoregressive sequence pre-training framework. Our approach various 3D medical images based on spatial, contrast, and semantic correlations, treating them as interconnected visual tokens within a token sequence.
  arXiv  Detail & Related papers  (2024-09-13T10:19:10Z)
• Cross-Dimensional Medical Self-Supervised Representation Learning Based on a Pseudo-3D Transformation [68.60747298865394]
  We propose a new cross-dimensional SSL framework based on a pseudo-3D transformation (CDSSL-P3D) Specifically, we introduce an image transformation based on the im2col algorithm, which converts 2D images into a format consistent with 3D data. This transformation enables seamless integration of 2D and 3D data, and facilitates cross-dimensional self-supervised learning for 3D medical image analysis.
  arXiv  Detail & Related papers  (2024-06-03T02:57:25Z)
• M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models [49.5030774873328]
  Previous research has primarily focused on 2D medical images, leaving 3D images under-explored, despite their richer spatial information. We present a large-scale 3D multi-modal medical dataset, M3D-Data, comprising 120K image-text pairs and 662K instruction-response pairs. We also introduce a new 3D multi-modal medical benchmark, M3D-Bench, which facilitates automatic evaluation across eight tasks.
  arXiv  Detail & Related papers  (2024-03-31T06:55:12Z)
• Generative Enhancement for 3D Medical Images [74.17066529847546]
  We propose GEM-3D, a novel generative approach to the synthesis of 3D medical images. Our method begins with a 2D slice, noted as the informed slice to serve the patient prior, and propagates the generation process using a 3D segmentation mask. By decomposing the 3D medical images into masks and patient prior information, GEM-3D offers a flexible yet effective solution for generating versatile 3D images.
  arXiv  Detail & Related papers  (2024-03-19T15:57:04Z)
• T3D: Towards 3D Medical Image Understanding through Vision-Language Pre-training [33.548818136506334]
  We introduce T3D, the first framework designed for high-resolution 3D medical images. T3D incorporates two text-informed pretext tasks: (lowerromannumeral1) text-informed contrastive learning; (lowerromannumeral2) text-informed image restoration. T3D significantly outperforms current vSSL methods in tasks like organ and tumor segmentation, as well as disease classification.
  arXiv  Detail & Related papers  (2023-12-03T23:03:22Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• Multi-View Vertebra Localization and Identification from CT Images [57.56509107412658]
  We propose a multi-view vertebra localization and identification from CT images. We convert the 3D problem into a 2D localization and identification task on different views. Our method can learn the multi-view global information naturally.
  arXiv  Detail & Related papers  (2023-07-24T14:43:07Z)
• MedGen3D: A Deep Generative Framework for Paired 3D Image and Mask Generation [17.373961762646356]
  We present MedGen3D, a framework that can generate paired 3D medical images and masks. Our proposed framework guarantees accurate alignment between synthetic images and segmentation maps.
  arXiv  Detail & Related papers  (2023-04-08T21:43:26Z)
• 3D Matting: A Benchmark Study on Soft Segmentation Method for Pulmonary Nodules Applied in Computed Tomography [32.775884701366465]
  We introduce the image matting into the 3D scenes and use the alpha matte, i.e., a soft mask, to describe lesions in a 3D medical image. To address this issue, we conduct a comprehensive study of 3D matting, including both traditional and deep-learning-based methods. We propose the first end-to-end deep 3D matting network and implement a solid 3D medical image matting benchmark.
  arXiv  Detail & Related papers  (2022-10-11T02:40:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.