Under the Cover Infant Pose Estimation using Multimodal Data

• URL: http://arxiv.org/abs/2210.00662v1
• Date: Mon, 3 Oct 2022 00:34:45 GMT
• Title: Under the Cover Infant Pose Estimation using Multimodal Data
• Authors: Daniel G. Kyrollos, Anthony Fuller, Kim Greenwood, JoAnn Harrold and James R. Green
• Abstract summary: We present a novel dataset, Simultaneously-collected multimodal Mannequin Lying pose (SMaL) dataset, for under the cover infant pose estimation. We successfully infer full body pose under the cover by training state-of-art pose estimation methods. Our best performing model was able to detect joints under the cover within 25mm 86% of the time with an overall mean error of 16.9mm.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Infant pose monitoring during sleep has multiple applications in both healthcare and home settings. In a healthcare setting, pose detection can be used for region of interest detection and movement detection for noncontact based monitoring systems. In a home setting, pose detection can be used to detect sleep positions which has shown to have a strong influence on multiple health factors. However, pose monitoring during sleep is challenging due to heavy occlusions from blanket coverings and low lighting. To address this, we present a novel dataset, Simultaneously-collected multimodal Mannequin Lying pose (SMaL) dataset, for under the cover infant pose estimation. We collect depth and pressure imagery of an infant mannequin in different poses under various cover conditions. We successfully infer full body pose under the cover by training state-of-art pose estimation methods and leveraging existing multimodal adult pose datasets for transfer learning. We demonstrate a hierarchical pretraining strategy for transformer-based models to significantly improve performance on our dataset. Our best performing model was able to detect joints under the cover within 25mm 86% of the time with an overall mean error of 16.9mm. Data, code and models publicly available at https://github.com/DanielKyr/SMaL

Related papers

• Comparison of marker-less 2D image-based methods for infant pose estimation [2.7726930707973048]
  The General Movement Assessment (GMA) is a video-based tool to classify infant motor functioning. We compare the performance of available generic- and infant-pose estimators, and the choice of viewing angle for optimal recordings. The results show that the best performing generic model trained on adults, ViTPose, also performs best on infants.
  arXiv  Detail & Related papers  (2024-10-07T12:21:49Z)
• PoseBench: Benchmarking the Robustness of Pose Estimation Models under Corruptions [57.871692507044344]
  Pose estimation aims to accurately identify anatomical keypoints in humans and animals using monocular images. Current models are typically trained and tested on clean data, potentially overlooking the corruption during real-world deployment. We introduce PoseBench, a benchmark designed to evaluate the robustness of pose estimation models against real-world corruption.
  arXiv  Detail & Related papers  (2024-06-20T14:40:17Z)
• AggPose: Deep Aggregation Vision Transformer for Infant Pose Estimation [6.9000851935487075]
  We propose infant pose dataset and Deep Aggregation Vision Transformer for human pose estimation. AggPose is a fast trained full transformer framework without using convolution operations to extract features in the early stages. We show that AggPose could effectively learn the multi-scale features among different resolutions and significantly improve the performance of infant pose estimation.
  arXiv  Detail & Related papers  (2022-05-11T05:34:14Z)
• Vogtareuth Rehab Depth Datasets: Benchmark for Marker-less Posture Estimation in Rehabilitation [55.41644538483948]
  We propose two rehabilitation-specific pose datasets containing depth images and 2D pose information of patients performing rehab exercises. We use a state-of-the-art marker-less posture estimation model which is trained on a non-rehab benchmark dataset. We show that our dataset can be used to train pose models to detect rehab-specific complex postures.
  arXiv  Detail & Related papers  (2021-08-23T16:18:26Z)
• Learning Dynamics via Graph Neural Networks for Human Pose Estimation and Tracking [98.91894395941766]
  We propose a novel online approach to learning the pose dynamics, which are independent of pose detections in current fame. Specifically, we derive this prediction of dynamics through a graph neural network(GNN) that explicitly accounts for both spatial-temporal and visual information. Experiments on PoseTrack 2017 and PoseTrack 2018 datasets demonstrate that the proposed method achieves results superior to the state of the art on both human pose estimation and tracking tasks.
  arXiv  Detail & Related papers  (2021-06-07T16:36:50Z)
• Unsupervised Human Pose Estimation through Transforming Shape Templates [2.729524133721473]
  We present a novel method for learning pose estimators for human adults and infants in an unsupervised fashion. We demonstrate the effectiveness of our approach on two different datasets including adults and infants.
  arXiv  Detail & Related papers  (2021-05-10T07:15:56Z)
• Multimodal In-bed Pose and Shape Estimation under the Blankets [77.12439296395733]
  We propose a pyramid scheme to fuse different modalities in a way that best leverages the knowledge captured by the multimodal sensors. We employ an attention-based reconstruction module to generate uncovered modalities, which are further fused to update current estimation.
  arXiv  Detail & Related papers  (2020-12-12T05:35:23Z)
• Invariant Representation Learning for Infant Pose Estimation with Small Data [14.91506452479778]
  We release a hybrid synthetic and real infant pose dataset with small yet diverse real images as well as generated synthetic infant poses. In our ablation study, with identical network structure, models trained on SyRIP dataset show noticeable improvement over the ones trained on the only other public infant pose datasets. One of our best infant pose estimation performers on the state-of-the-art DarkPose model shows mean average precision (mAP) of 93.6.
  arXiv  Detail & Related papers  (2020-10-13T01:10:14Z)
• Simultaneously-Collected Multimodal Lying Pose Dataset: Towards In-Bed Human Pose Monitoring under Adverse Vision Conditions [15.12849597272402]
  In-bed human pose estimation has significant values in many healthcare applications. In this paper, we introduce our Simultaneously-collected multimodal Lying Pose dataset. We show that state-of-the-art 2D pose estimation models can be trained effectively with SLP data with promising performance as high as 95% at PCKh@0.5 on a single modality.
  arXiv  Detail & Related papers  (2020-08-20T02:20:35Z)
• 6D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inference [67.70859730448473]
  We present a multimodal camera relocalization framework that captures ambiguities and uncertainties. We predict multiple camera pose hypotheses as well as the respective uncertainty for each prediction. We introduce a new dataset specifically designed to foster camera localization research in ambiguous environments.
  arXiv  Detail & Related papers  (2020-04-09T20:55:06Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.