From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction

• URL: http://arxiv.org/abs/2505.03599v1
• Date: Tue, 06 May 2025 15:01:43 GMT
• Title: From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction
• Authors: Fengming Lin, Arezoo Zakeri, Yidan Xue, Michael MacRaild, Haoran Dou, Zherui Zhou, Ziwei Zou, Ali Sarrami-Foroushani, Jinming Duan, Alejandro F. Frangi,
• Abstract summary: This survey systematically categorizes existing approaches into four main categories: template models, statistical models, generative models, and implicit models.<n>We provide an extensive evaluation of these methods across various anatomical applications, from cardiac imaging to neurological studies.<n>The survey identifies current challenges in the field, including requirements for topological correctness, geometric accuracy, and multi-modality integration.
• Score: 38.67693323186832
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning-based medical image-to-mesh reconstruction has rapidly evolved, enabling the transformation of medical imaging data into three-dimensional mesh models that are critical in computational medicine and in silico trials for advancing our understanding of disease mechanisms, and diagnostic and therapeutic techniques in modern medicine. This survey systematically categorizes existing approaches into four main categories: template models, statistical models, generative models, and implicit models. Each category is analysed in detail, examining their methodological foundations, strengths, limitations, and applicability to different anatomical structures and imaging modalities. We provide an extensive evaluation of these methods across various anatomical applications, from cardiac imaging to neurological studies, supported by quantitative comparisons using standard metrics. Additionally, we compile and analyze major public datasets available for medical mesh reconstruction tasks and discuss commonly used evaluation metrics and loss functions. The survey identifies current challenges in the field, including requirements for topological correctness, geometric accuracy, and multi-modality integration. Finally, we present promising future research directions in this domain. This systematic review aims to serve as a comprehensive reference for researchers and practitioners in medical image analysis and computational medicine.

Related papers

• A Survey of Medical Point Cloud Shape Learning: Registration, Reconstruction and Variation [1.249743555715438]
  Point clouds have become an increasingly important representation for 3D medical imaging, offering a compact, surface-preserving alternative to traditional voxel or mesh-based approaches.<n>Recent advances in deep learning have enabled rapid progress in extracting, modeling, and analyzing anatomical shapes directly from point cloud data.
  arXiv  Detail & Related papers  (2025-08-05T04:04:20Z)
• A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models [28.34025112894094]
  The review focuses on two main categories: task-specific multimodal approaches and large-scale multimodal foundation models.<n>The survey critically examines important datasets, evaluation metrics, and methodological innovations.<n>It also discusses ongoing challenges such as variability in data, limited annotations, lack of interpretability, and issues with generalizability across different patient populations.
  arXiv  Detail & Related papers  (2025-07-31T10:49:21Z)
• PyTDC: A multimodal machine learning training, evaluation, and inference platform for biomedical foundation models [59.17570021208177]
  PyTDC is a machine-learning platform providing streamlined training, evaluation, and inference software for multimodal biological AI models.<n>This paper discusses the components of PyTDC's architecture and, to our knowledge, the first-of-its-kind case study on the introduced single-cell drug-target nomination ML task.
  arXiv  Detail & Related papers  (2025-05-08T18:15:38Z)
• A 3D Facial Reconstruction Evaluation Methodology: Comparing Smartphone Scans with Deep Learning Based Methods Using Geometry and Morphometry Criteria [60.865754842465684]
  Three-dimensional (3D) facial shape analysis has gained interest due to its potential clinical applications.<n>High cost of advanced 3D facial acquisition systems limits their widespread use, driving the development of low-cost acquisition and reconstruction methods.<n>This study introduces a novel evaluation methodology that goes beyond traditional geometry-based benchmarks by integrating morphometric shape analysis techniques.
  arXiv  Detail & Related papers  (2025-02-13T15:47:45Z)
• A Comprehensive Review of U-Net and Its Variants: Advances and Applications in Medical Image Segmentation [0.0]
  This paper classifies medical image datasets on the basis of their imaging modalities and examines U-Net and its various improvement models.<n>We summarize the four central improvement mechanisms of the U-Net and U-Net variant algorithms.<n>We propose potential avenues and strategies for future advancements.
  arXiv  Detail & Related papers  (2025-02-09T13:11:51Z)
• Anatomy-guided Pathology Segmentation [56.883822515800205]
  We develop a generalist segmentation model that combines anatomical and pathological information, aiming to enhance the segmentation accuracy of pathological features. Our Anatomy-Pathology Exchange (APEx) training utilizes a query-based segmentation transformer which decodes a joint feature space into query-representations for human anatomy. In doing so, we are able to report the best results across the board on FDG-PET-CT and Chest X-Ray pathology segmentation tasks with a margin of up to 3.3% as compared to strong baseline methods.
  arXiv  Detail & Related papers  (2024-07-08T11:44:15Z)
• QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge [93.61262892578067]
  Uncertainty in medical image segmentation tasks, especially inter-rater variability, presents a significant challenge. This variability directly impacts the development and evaluation of automated segmentation algorithms. We report the set-up and summarize the benchmark results of the Quantification of Uncertainties in Biomedical Image Quantification Challenge (QUBIQ)
  arXiv  Detail & Related papers  (2024-03-19T17:57:24Z)
• Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision [6.2847894163744105]
  Foundation models are large-scale, pre-trained deep-learning models adapted to a wide range of downstream tasks. These models facilitate contextual reasoning, generalization, and prompt capabilities at test time. Capitalizing on the advances in computer vision, medical imaging has also marked a growing interest in these models.
  arXiv  Detail & Related papers  (2023-10-28T12:08:12Z)
• MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification [41.16626194300303]
  Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications. Recent advances further enable adapting foundation models in downstream tasks efficiently using only a few training samples. Yet, the application of such learning paradigms in medical image analysis remains scarce due to the shortage of publicly accessible data and benchmarks.
  arXiv  Detail & Related papers  (2023-06-16T01:46:07Z)
• Explainable Deep Learning Methods in Medical Image Classification: A Survey [0.0]
  State-of-the-art deep learning models have achieved human-level accuracy on the classification of different types of medical data. These models are hardly adopted in clinical, mainly due to their lack of interpretability. The black-box-ness of deep learning models has raised the need for devising strategies to explain the decision process of these models.
  arXiv  Detail & Related papers  (2022-05-10T09:28:14Z)
• Domain Shift in Computer Vision models for MRI data analysis: An Overview [64.69150970967524]
  Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use. Poor transferability of themodels to data from different sources or acquisition domains is one of the reasons for that.
  arXiv  Detail & Related papers  (2020-10-14T16:34:21Z)
• Benchmarking off-the-shelf statistical shape modeling tools in clinical applications [53.47202621511081]
  We systematically assess the outcome of widely used, state-of-the-art SSM tools. We propose validation frameworks for anatomical landmark/measurement inference and lesion screening. ShapeWorks and Deformetrica shape models are found to capture clinically relevant population-level variability.
  arXiv  Detail & Related papers  (2020-09-07T03:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.