FetusMap: Fetal Pose Estimation in 3D Ultrasound

• URL: http://arxiv.org/abs/1910.04935v2
• Date: Sun, 3 Mar 2024 12:08:37 GMT
• Title: FetusMap: Fetal Pose Estimation in 3D Ultrasound
• Authors: Xin Yang, Wenlong Shi, Haoran Dou, Jikuan Qian, Yi Wang, Wufeng Xue, Shengli Li, Dong Ni, Pheng-Ann Heng
• Abstract summary: We propose to estimate the 3D pose of fetus in US volumes to facilitate its quantitative analyses. This is the first work about 3D pose estimation of fetus in the literature. We propose a self-supervised learning (SSL) framework to finetune the deep network to form visually plausible pose predictions.
• Score: 42.59502360552173
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The 3D ultrasound (US) entrance inspires a multitude of automated prenatal examinations. However, studies about the structuralized description of the whole fetus in 3D US are still rare. In this paper, we propose to estimate the 3D pose of fetus in US volumes to facilitate its quantitative analyses in global and local scales. Given the great challenges in 3D US, including the high volume dimension, poor image quality, symmetric ambiguity in anatomical structures and large variations of fetal pose, our contribution is three-fold. (i) This is the first work about 3D pose estimation of fetus in the literature. We aim to extract the skeleton of whole fetus and assign different segments/joints with correct torso/limb labels. (ii) We propose a self-supervised learning (SSL) framework to finetune the deep network to form visually plausible pose predictions. Specifically, we leverage the landmark-based registration to effectively encode case-adaptive anatomical priors and generate evolving label proxy for supervision. (iii) To enable our 3D network perceive better contextual cues with higher resolution input under limited computing resource, we further adopt the gradient check-pointing (GCP) strategy to save GPU memory and improve the prediction. Extensively validated on a large 3D US dataset, our method tackles varying fetal poses and achieves promising results. 3D pose estimation of fetus has potentials in serving as a map to provide navigation for many advanced studies.

Related papers

• Geometric Transformation Uncertainty for Improving 3D Fetal Brain Pose Prediction from Freehand 2D Ultrasound Videos [0.8579241568505183]
  We propose an uncertainty-aware deep learning model for automated 3D plane localization in 2D fetal brain images. Our proposed method, QAERTS, demonstrates superior pose estimation accuracy than the state-of-the-art and most of the uncertainty-based approaches.
  arXiv  Detail & Related papers  (2024-05-21T22:42:08Z)
• FetusMapV2: Enhanced Fetal Pose Estimation in 3D Ultrasound [28.408626329596668]
  We propose a novel 3D fetal pose estimation framework (called FetusMapV2) to overcome the above challenges. First, we propose a scheme that explores the complementary network structure-unconstrained and activation-unreserved GPU memory management approaches. Second, we design a novel Pair Loss to mitigate confusion caused by symmetrical and similar anatomical structures. Third, we propose a shape priors-based self-supervised learning by selecting the relatively stable landmarks to refine the pose online.
  arXiv  Detail & Related papers  (2023-10-30T06:18:47Z)
• FPUS23: An Ultrasound Fetus Phantom Dataset with Deep Neural Network Evaluations for Fetus Orientations, Fetal Planes, and Anatomical Features [10.404128105946583]
  We present a novel fetus phantom ultrasound dataset, FPUS23, which can be used to identify the correct diagnostic planes for estimating fetal biometric values. The entire dataset is composed of 15,728 images, which are used to train four different Deep Neural Network models. We have also evaluated the models trained using our FPUS23 dataset, to show that the information learned by these models can be used to substantially increase the accuracy on real-world ultrasound fetus datasets.
  arXiv  Detail & Related papers  (2023-03-14T12:46:48Z)
• Ultrasound Plane Pose Regression: Assessing Generalized Pose Coordinates in the Fetal Brain [9.465965149145559]
  We aim to build a US plane localization system for 3D visualization, training, and guidance without integrating additional sensors. This work builds on our previous work, which predicts the six-dimensional (6D) pose of arbitrarily oriented US planes slicing the fetal brain. We investigate the impact of registration quality in the training and testing data and its subsequent effect on trained models.
  arXiv  Detail & Related papers  (2023-01-19T21:16:36Z)
• Agent with Tangent-based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound [56.7645826576439]
  We introduce a novel reinforcement learning framework for automatic SP localization in 3D US. First, we formulate SP localization in 3D US as a tangent-point-based problem in RL to restructure the action space. Second, we design an auxiliary task learning strategy to enhance the model's ability to recognize subtle differences crossing Non-SPs and SPs in plane search.
  arXiv  Detail & Related papers  (2022-07-01T14:53:27Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Hybrid Attention for Automatic Segmentation of Whole Fetal Head in Prenatal Ultrasound Volumes [52.53375964591765]
  We propose the first fully-automated solution to segment the whole fetal head in US volumes. The segmentation task is firstly formulated as an end-to-end volumetric mapping under an encoder-decoder deep architecture. We then combine the segmentor with a proposed hybrid attention scheme (HAS) to select discriminative features and suppress the non-informative volumetric features.
  arXiv  Detail & Related papers  (2020-04-28T14:43:05Z)
• Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation [107.07047303858664]
  Large-scale human datasets with 3D ground-truth annotations are difficult to obtain in the wild. We address this problem by augmenting existing 2D datasets with high-quality 3D pose fits. The resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks.
  arXiv  Detail & Related papers  (2020-04-07T20:21:18Z)
• Region Proposal Network with Graph Prior and IoU-Balance Loss for Landmark Detection in 3D Ultrasound [16.523977092204813]
  3D ultrasound (US) can facilitate detailed prenatal examinations for fetal growth monitoring. To analyze a 3D US volume, it is fundamental to identify anatomical landmarks accurately. We exploit an object detection framework to detect landmarks in 3D fetal facial US volumes.
  arXiv  Detail & Related papers  (2020-04-01T03:00:03Z)
• Anatomy-aware 3D Human Pose Estimation with Bone-based Pose Decomposition [92.99291528676021]
  Instead of directly regressing the 3D joint locations, we decompose the task into bone direction prediction and bone length prediction. Our motivation is the fact that the bone lengths of a human skeleton remain consistent across time. Our full model outperforms the previous best results on Human3.6M and MPI-INF-3DHP datasets.
  arXiv  Detail & Related papers  (2020-02-24T15:49:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.