Real-time, low-cost multi-person 3D pose estimation

• URL: http://arxiv.org/abs/2110.11414v1
• Date: Mon, 11 Oct 2021 12:42:00 GMT
• Title: Real-time, low-cost multi-person 3D pose estimation
• Authors: Alice Ruget, Max Tyler, Germ\'an Mora Mart\'in, Stirling Scholes, Feng Zhu, Istvan Gyongy, Brent Hearn, Steve McLaughlin, Abderrahim Halimi, Jonathan Leach
• Abstract summary: Three-dimensional pose estimation traditionally requires advanced equipment, such as multiple linked intensity cameras or high-resolution time-of-flight cameras to produce depth images. Here, we demonstrate that computational imaging methods can achieve accurate pose estimation and overcome the apparent limitations of time-of-flight sensors designed for much simpler tasks. This work opens up promising real-life applications in scenarios that were previously restricted by the advanced hardware requirements and cost of time-of-flight technology.
• Score: 8.093696116585717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The process of tracking human anatomy in computer vision is referred to pose estimation, and it is used in fields ranging from gaming to surveillance. Three-dimensional pose estimation traditionally requires advanced equipment, such as multiple linked intensity cameras or high-resolution time-of-flight cameras to produce depth images. However, there are applications, e.g.~consumer electronics, where significant constraints are placed on the size, power consumption, weight and cost of the usable technology. Here, we demonstrate that computational imaging methods can achieve accurate pose estimation and overcome the apparent limitations of time-of-flight sensors designed for much simpler tasks. The sensor we use is already widely integrated in consumer-grade mobile devices, and despite its low spatial resolution, only 4$\times$4 pixels, our proposed Pixels2Pose system transforms its data into accurate depth maps and 3D pose data of multiple people up to a distance of 3 m from the sensor. We are able to generate depth maps at a resolution of 32$\times$32 and 3D localization of a body parts with an error of only $\approx$10 cm at a frame rate of 7 fps. This work opens up promising real-life applications in scenarios that were previously restricted by the advanced hardware requirements and cost of time-of-flight technology.

Related papers

• Ultra Inertial Poser: Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Ranging [23.010268017856824]
  We propose Ultra Inertial Poser, a novel 3D full body pose estimation method. Our method constrains drift and jitter in inertial tracking via inter-sensor distances. Our experiments show state-of-the-art performance for our method over PIP and TIP.
  arXiv  Detail & Related papers  (2024-04-30T13:14:11Z)
• EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams [59.77837807004765]
  This paper introduces a new problem, i.e., 3D human motion capture from an egocentric monocular event camera with a fisheye lens. Event streams have high temporal resolution and provide reliable cues for 3D human motion capture under high-speed human motions and rapidly changing illumination. Our EE3D demonstrates robustness and superior 3D accuracy compared to existing solutions while supporting real-time 3D pose update rates of 140Hz.
  arXiv  Detail & Related papers  (2024-04-12T17:59:47Z)
• Prototipo de un Contador Bidireccional Automático de Personas basado en sensores de visión 3D [39.58317527488534]
  3D sensors, also known as RGB-D sensors, utilize depth images where each pixel measures the distance from the camera to objects. The described prototype uses RGB-D sensors for bidirectional people counting in venues, aiding security and surveillance in spaces like stadiums or airports. The system includes a RealSense D415 depth camera and a mini-computer running object detection algorithms to count people and a 2D camera for identity verification.
  arXiv  Detail & Related papers  (2024-03-18T23:18:40Z)
• Improving Real-Time Omnidirectional 3D Multi-Person Human Pose Estimation with People Matching and Unsupervised 2D-3D Lifting [3.231937990387248]
  Current human pose estimation systems focus on retrieving an accurate 3D global estimate of a single person. This paper presents one of the first 3D multi-person human pose estimation systems that is able to work in real-time.
  arXiv  Detail & Related papers  (2024-03-14T14:30:31Z)
• Multi-Modal Dataset Acquisition for Photometrically Challenging Object [56.30027922063559]
  This paper addresses the limitations of current datasets for 3D vision tasks in terms of accuracy, size, realism, and suitable imaging modalities for photometrically challenging objects. We propose a novel annotation and acquisition pipeline that enhances existing 3D perception and 6D object pose datasets.
  arXiv  Detail & Related papers  (2023-08-21T10:38:32Z)
• Economical Quaternion Extraction from a Human Skeletal Pose Estimate using 2-D Cameras [0.0]
  The proposed algorithm extracts a quaternion from a 2-D frame capturing an image of a human object at a sub-fifty millisecond latency. The algorithm seeks to bypass the funding barrier and improve accessibility for robotics researchers involved in designing control systems.
  arXiv  Detail & Related papers  (2023-03-15T14:41:17Z)
• E$^3$Pose: Energy-Efficient Edge-assisted Multi-camera System for Multi-human 3D Pose Estimation [5.50767672740241]
  Multi-human 3D pose estimation plays a key role in establishing a seamless connection between the real world and the virtual world. We propose an energy-efficient edge-assisted multiple-camera system, dubbed E$3$Pose, for real-time multi-human 3D pose estimation. Our results show that a significant energy saving (up to 31.21%) can be achieved while maintaining a high 3D pose estimation accuracy comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-01-21T21:53:33Z)
• Scene-Aware 3D Multi-Human Motion Capture from a Single Camera [83.06768487435818]
  We consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera. We leverage recent advances in computer vision using large-scale pre-trained models for a variety of modalities, including 2D body joints, joint angles, normalized disparity maps, and human segmentation masks. In particular, we estimate the scene depth and unique person scale from normalized disparity predictions using the 2D body joints and joint angles.
  arXiv  Detail & Related papers  (2023-01-12T18:01:28Z)
• MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision [72.5863451123577]
  We show how to train a neural model that can perform accurate 3D pose and camera estimation. Our method outperforms both classical bundle adjustment and weakly-supervised monocular 3D baselines.
  arXiv  Detail & Related papers  (2021-08-10T18:39:56Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)
• Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation [57.11299763566534]
  We present a solution to recover 3D pose from multi-view images captured with spatially calibrated cameras. We exploit 3D geometry to fuse input images into a unified latent representation of pose, which is disentangled from camera view-points. Our architecture then conditions the learned representation on camera projection operators to produce accurate per-view 2d detections.
  arXiv  Detail & Related papers  (2020-04-05T12:52:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.