PeopleSansPeople: A Synthetic Data Generator for Human-Centric Computer Vision

• URL: http://arxiv.org/abs/2112.09290v1
• Date: Fri, 17 Dec 2021 02:33:31 GMT
• Title: PeopleSansPeople: A Synthetic Data Generator for Human-Centric Computer Vision
• Authors: Salehe Erfanian Ebadi, You-Cyuan Jhang, Alex Zook, Saurav Dhakad, Adam Crespi, Pete Parisi, Steven Borkman, Jonathan Hogins, Sujoy Ganguly
• Abstract summary: We release a human-centric synthetic data generator PeopleSansPeople. It contains simulation-ready 3D human assets, a parameterized lighting and camera system, and generates 2D and 3D bounding box, instance and semantic segmentation, and COCO pose labels.
• Score: 3.5694949627557846
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, person detection and human pose estimation have made great strides, helped by large-scale labeled datasets. However, these datasets had no guarantees or analysis of human activities, poses, or context diversity. Additionally, privacy, legal, safety, and ethical concerns may limit the ability to collect more human data. An emerging alternative to real-world data that alleviates some of these issues is synthetic data. However, creation of synthetic data generators is incredibly challenging and prevents researchers from exploring their usefulness. Therefore, we release a human-centric synthetic data generator PeopleSansPeople which contains simulation-ready 3D human assets, a parameterized lighting and camera system, and generates 2D and 3D bounding box, instance and semantic segmentation, and COCO pose labels. Using PeopleSansPeople, we performed benchmark synthetic data training using a Detectron2 Keypoint R-CNN variant [1]. We found that pre-training a network using synthetic data and fine-tuning on target real-world data (few-shot transfer to limited subsets of COCO-person train [2]) resulted in a keypoint AP of $60.37 \pm 0.48$ (COCO test-dev2017) outperforming models trained with the same real data alone (keypoint AP of $55.80$) and pre-trained with ImageNet (keypoint AP of $57.50$). This freely-available data generator should enable a wide range of research into the emerging field of simulation to real transfer learning in the critical area of human-centric computer vision.

Related papers

• WheelPose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Users [5.057643544417776]
  Existing pose estimation models perform poorly on wheelchair users due to a lack of representation in training data. We present a data synthesis pipeline to address this disparity in data collection. Our pipeline generates synthetic data of wheelchair users using motion capture data and motion generation outputs simulated in the Unity game engine.
  arXiv  Detail & Related papers  (2024-04-25T22:17:32Z)
• Learning Human Action Recognition Representations Without Real Humans [66.61527869763819]
  We present a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model. We then evaluate the transferability of the representation learned on this data to a diverse set of downstream action recognition benchmarks. Our approach outperforms previous baselines by up to 5%.
  arXiv  Detail & Related papers  (2023-11-10T18:38:14Z)
• Synthetic data, real errors: how (not) to publish and use synthetic data [86.65594304109567]
  We show how the generative process affects the downstream ML task. We introduce Deep Generative Ensemble (DGE) to approximate the posterior distribution over the generative process model parameters.
  arXiv  Detail & Related papers  (2023-05-16T07:30:29Z)
• Synthetic Data-based Detection of Zebras in Drone Imagery [0.8249180979158817]
  We present an approach for training an animal detector using only synthetic data. The dataset includes RGB, depth, skeletal joint locations, pose, shape and instance segmentations for each subject. We show that we can detect zebras by using only synthetic data during training.
  arXiv  Detail & Related papers  (2023-04-30T09:24:31Z)
• SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling [93.60731530276911]
  We introduce a new synthetic dataset, SynBody, with three appealing features. The dataset comprises 1.2M images with corresponding accurate 3D annotations, covering 10,000 human body models, 1,187 actions, and various viewpoints.
  arXiv  Detail & Related papers  (2023-03-30T13:30:12Z)
• A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation [42.2398858786125]
  Deep learning in computer vision has achieved great success with the price of large-scale labeled training data. The uncontrollable data collection process produces non-IID training and test data, where undesired duplication may exist. To circumvent them, an alternative is to generate synthetic data via 3D rendering with domain randomization.
  arXiv  Detail & Related papers  (2023-03-16T09:03:52Z)
• Video-based Pose-Estimation Data as Source for Transfer Learning in Human Activity Recognition [71.91734471596433]
  Human Activity Recognition (HAR) using on-body devices identifies specific human actions in unconstrained environments. Previous works demonstrated that transfer learning is a good strategy for addressing scenarios with scarce data. This paper proposes using datasets intended for human-pose estimation as a source for transfer learning.
  arXiv  Detail & Related papers  (2022-12-02T18:19:36Z)
• GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation [60.07812405063708]
  3D point cloud semantic segmentation is fundamental for autonomous driving. Most approaches in the literature neglect an important aspect, i.e., how to deal with domain shift when handling dynamic scenes. This paper advances the state of the art in this research field.
  arXiv  Detail & Related papers  (2022-07-20T09:06:07Z)
• Hands-Up: Leveraging Synthetic Data for Hands-On-Wheel Detection [0.38233569758620045]
  This work demonstrates the use of synthetic photo-realistic in-cabin data to train a Driver Monitoring System. We show how performing error analysis and generating the missing edge-cases in our platform boosts performance. This showcases the ability of human-centric synthetic data to generalize well to the real world.
  arXiv  Detail & Related papers  (2022-05-31T23:34:12Z)
• StyleGAN-Human: A Data-Centric Odyssey of Human Generation [96.7080874757475]
  This work takes a data-centric perspective and investigates multiple critical aspects in "data engineering" We collect and annotate a large-scale human image dataset with over 230K samples capturing diverse poses and textures. We rigorously investigate three essential factors in data engineering for StyleGAN-based human generation, namely data size, data distribution, and data alignment.
  arXiv  Detail & Related papers  (2022-04-25T17:55:08Z)
• Fake It Till You Make It: Face analysis in the wild using synthetic data alone [9.081019005437309]
  We show that it is possible to perform face-related computer vision in the wild using synthetic data alone. We describe how to combine a procedurally-generated 3D face model with a comprehensive library of hand-crafted assets to render training images with unprecedented realism.
  arXiv  Detail & Related papers  (2021-09-30T13:07:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.