Semi-supervised Body Parsing and Pose Estimation for Enhancing Infant General Movement Assessment

• URL: http://arxiv.org/abs/2210.08054v1
• Date: Fri, 14 Oct 2022 18:46:30 GMT
• Title: Semi-supervised Body Parsing and Pose Estimation for Enhancing Infant General Movement Assessment
• Authors: Haomiao Ni, Yuan Xue, Liya Ma, Qian Zhang, Xiaoye Li, Xiaolei Huang
• Abstract summary: General movement assessment (GMA) of infant movement videos (IMVs) is an effective method for early detection of cerebral palsy (CP) in infants. We demonstrate in this paper that end-to-end trainable neural networks for image sequence recognition can be applied to achieve good results in GMA. We propose a semi-supervised model, termed SiamParseNet (SPN), which consists of two branches, one for intra-frame body parts segmentation and another for inter-frame label propagation.
• Score: 11.33138866472943
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: General movement assessment (GMA) of infant movement videos (IMVs) is an effective method for early detection of cerebral palsy (CP) in infants. We demonstrate in this paper that end-to-end trainable neural networks for image sequence recognition can be applied to achieve good results in GMA, and more importantly, augmenting raw video with infant body parsing and pose estimation information can significantly improve performance. To solve the problem of efficiently utilizing partially labeled IMVs for body parsing, we propose a semi-supervised model, termed SiamParseNet (SPN), which consists of two branches, one for intra-frame body parts segmentation and another for inter-frame label propagation. During training, the two branches are jointly trained by alternating between using input pairs of only labeled frames and input of both labeled and unlabeled frames. We also investigate training data augmentation by proposing a factorized video generative adversarial network (FVGAN) to synthesize novel labeled frames for training. When testing, we employ a multi-source inference mechanism, where the final result for a test frame is either obtained via the segmentation branch or via propagation from a nearby key frame. We conduct extensive experiments for body parsing using SPN on two infant movement video datasets, where SPN coupled with FVGAN achieves state-of-the-art performance. We further demonstrate that SPN can be easily adapted to the infant pose estimation task with superior performance. Last but not least, we explore the clinical application of our method for GMA. We collected a new clinical IMV dataset with GMA annotations, and our experiments show that SPN models for body parsing and pose estimation trained on the first two datasets generalize well to the new clinical dataset and their results can significantly boost the CRNN-based GMA prediction performance.

Related papers

• Intrapartum Ultrasound Image Segmentation of Pubic Symphysis and Fetal Head Using Dual Student-Teacher Framework with CNN-ViT Collaborative Learning [1.5233179662962222]
  The segmentation of the pubic symphysis and fetal head (PSFH) constitutes a pivotal step in monitoring labor progression and identifying potential delivery complications. Traditional semi-supervised learning approaches primarily utilize a unified network model based on Convolutional Neural Networks (CNNs) We introduce a novel framework, the Dual-Student and Teacher Combining CNN and Transformer (DSTCT)
  arXiv  Detail & Related papers  (2024-09-11T00:57:31Z)
• Measuring proximity to standard planes during fetal brain ultrasound scanning [8.328549443700858]
  This paper introduces a novel pipeline designed to bring ultrasound (US) plane pose estimation closer to clinical use. We propose a semi-supervised segmentation model utilizing both labeled SPs and unlabeled 3D US volume slices. Our model enables reliable segmentation across a diverse set of fetal brain images.
  arXiv  Detail & Related papers  (2024-04-10T16:04:21Z)
• Predicting Infant Brain Connectivity with Federated Multi-Trajectory GNNs using Scarce Data [54.55126643084341]
  Existing deep learning solutions suffer from three major limitations. We introduce FedGmTE-Net++, a federated graph-based multi-trajectory evolution network. Using the power of federation, we aggregate local learnings among diverse hospitals with limited datasets.
  arXiv  Detail & Related papers  (2024-01-01T10:20:01Z)
• Domain Adaptive Synapse Detection with Weak Point Annotations [63.97144211520869]
  We present AdaSyn, a framework for domain adaptive synapse detection with weak point annotations. In the WASPSYN challenge at I SBI 2023, our method ranks the 1st place.
  arXiv  Detail & Related papers  (2023-08-31T05:05:53Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Unsupervised Domain Adaptation Learning for Hierarchical Infant Pose Recognition with Synthetic Data [28.729049747477085]
  We present a CNN-based model which takes any infant image as input and predicts the coarse and fine-level pose labels. Our experimental results show that the proposed method can significantly align the distribution of synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2022-05-04T04:59:26Z)
• Joint-bone Fusion Graph Convolutional Network for Semi-supervised Skeleton Action Recognition [65.78703941973183]
  We propose a novel correlation-driven joint-bone fusion graph convolutional network (CD-JBF-GCN) as an encoder and use a pose prediction head as a decoder. Specifically, the CD-JBF-GC can explore the motion transmission between the joint stream and the bone stream. The pose prediction based auto-encoder in the self-supervised training stage allows the network to learn motion representation from unlabeled data.
  arXiv  Detail & Related papers  (2022-02-08T16:03:15Z)
• Cascaded Robust Learning at Imperfect Labels for Chest X-ray Segmentation [61.09321488002978]
  We present a novel cascaded robust learning framework for chest X-ray segmentation with imperfect annotation. Our model consists of three independent network, which can effectively learn useful information from the peer networks. Our methods could achieve a significant improvement on the accuracy in segmentation tasks compared to the previous methods.
  arXiv  Detail & Related papers  (2021-04-05T15:50:16Z)
• An Uncertainty-Driven GCN Refinement Strategy for Organ Segmentation [53.425900196763756]
  We propose a segmentation refinement method based on uncertainty analysis and graph convolutional networks. We employ the uncertainty levels of the convolutional network in a particular input volume to formulate a semi-supervised graph learning problem. We show that our method outperforms the state-of-the-art CRF refinement method by improving the dice score by 1% for the pancreas and 2% for spleen.
  arXiv  Detail & Related papers  (2020-12-06T18:55:07Z)
• Brain Stroke Lesion Segmentation Using Consistent Perception Generative Adversarial Network [22.444373004248217]
  Consistent PerceptionGenerative Adversarial Network (CPGAN) is proposed for semi-supervised stroke lesion segmentation. A similarity connection module (SCM) is designed to capture the information of multi-scale features. An assistant network is constructed to encourage the discriminator to learn meaningful feature representations.
  arXiv  Detail & Related papers  (2020-08-30T07:42:47Z)
• SiamParseNet: Joint Body Parsing and Label Propagation in Infant Movement Videos [12.99371655893686]
  General movement assessment (GMA) of infant movement videos (IMVs) is an effective method for the early detection of cerebral palsy (CP) in infants. We propose a semi-supervised body parsing model, termed SiamParseNet (SPN), to jointly learn single frame body parsing and label propagation between frames in a semi-supervised fashion.
  arXiv  Detail & Related papers  (2020-07-16T21:14:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.