Monocular Person Localization under Camera Ego-motion

• URL: http://arxiv.org/abs/2503.02916v1
• Date: Tue, 04 Mar 2025 11:07:27 GMT
• Title: Monocular Person Localization under Camera Ego-motion
• Authors: Yu Zhan, Hanjing Ye, Hong Zhang,
• Abstract summary: We consider person localization as a part of a pose estimation problem.<n>By representing a human with a four-point model, our method jointly estimates the 2D camera attitude and the person's 3D location.<n>Our method is further implemented into a person-following system and deployed on an agile quadruped robot.
• Score: 5.030357146921396
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Localizing a person from a moving monocular camera is critical for Human-Robot Interaction (HRI). To estimate the 3D human position from a 2D image, existing methods either depend on the geometric assumption of a fixed camera or use a position regression model trained on datasets containing little camera ego-motion. These methods are vulnerable to fierce camera ego-motion, resulting in inaccurate person localization. We consider person localization as a part of a pose estimation problem. By representing a human with a four-point model, our method jointly estimates the 2D camera attitude and the person's 3D location through optimization. Evaluations on both public datasets and real robot experiments demonstrate our method outperforms baselines in person localization accuracy. Our method is further implemented into a person-following system and deployed on an agile quadruped robot.

Related papers

• Reconstructing People, Places, and Cameras [57.81696692335401]
  "Humans and Structure from Motion" (HSfM) is a method for jointly reconstructing multiple human meshes, scene point clouds, and camera parameters in a metric world coordinate system. Our results show that incorporating human data into the SfM pipeline improves camera pose estimation.
  arXiv  Detail & Related papers  (2024-12-23T18:58:34Z)
• Exploring 3D Human Pose Estimation and Forecasting from the Robot's Perspective: The HARPER Dataset [52.22758311559]
  We introduce HARPER, a novel dataset for 3D body pose estimation and forecast in dyadic interactions between users and Spot. The key-novelty is the focus on the robot's perspective, i.e., on the data captured by the robot's sensors. The scenario underlying HARPER includes 15 actions, of which 10 involve physical contact between the robot and users.
  arXiv  Detail & Related papers  (2024-03-21T14:53:50Z)
• TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments [106.80978555346958]
  Current methods can't reliably estimate moving humans in global coordinates. TRACE is the first one-stage method to jointly recover and track 3D humans in global coordinates from dynamic cameras. It achieves state-of-the-art performance on tracking and HPS benchmarks.
  arXiv  Detail & Related papers  (2023-06-05T13:00:44Z)
• External Camera-based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors [22.5939915003931]
  We present an approach for estimating a mobile robot's pose w.r.t. the allocentric coordinates of a network of static cameras using multi-view RGB images. The images are processed online, locally on smart edge sensors by deep neural networks to detect the robot. With the robot's pose precisely estimated, its observations can be fused into the allocentric scene model.
  arXiv  Detail & Related papers  (2023-03-07T11:03:33Z)
• Scene-Aware 3D Multi-Human Motion Capture from a Single Camera [83.06768487435818]
  We consider the problem of estimating the 3D position of multiple humans in a scene as well as their body shape and articulation from a single RGB video recorded with a static camera. We leverage recent advances in computer vision using large-scale pre-trained models for a variety of modalities, including 2D body joints, joint angles, normalized disparity maps, and human segmentation masks. In particular, we estimate the scene depth and unique person scale from normalized disparity predictions using the 2D body joints and joint angles.
  arXiv  Detail & Related papers  (2023-01-12T18:01:28Z)
• 3D Human Pose Estimation in Multi-View Operating Room Videos Using Differentiable Camera Projections [2.486571221735935]
  We propose to directly optimise for localisation in 3D by training 2D CNNs end-to-end based on a 3D loss. Using videos from the MVOR dataset, we show that this end-to-end approach outperforms optimisation in 2D space.
  arXiv  Detail & Related papers  (2022-10-21T09:00:02Z)
• Embodied Scene-aware Human Pose Estimation [25.094152307452]
  We propose embodied scene-aware human pose estimation. Our method is one stage, causal, and recovers global 3D human poses in a simulated environment.
  arXiv  Detail & Related papers  (2022-06-18T03:50:19Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)
• Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution [34.301501457959056]
  We propose a temporal regression network with a gated convolution module to transform 2D joints to 3D. A simple yet effective localization approach is also conducted to transform the normalized pose to the global trajectory. Our proposed method outperforms most state-of-the-art 2D-to-3D pose estimation methods.
  arXiv  Detail & Related papers  (2020-10-31T04:35:24Z)
• Perceiving Humans: from Monocular 3D Localization to Social Distancing [93.03056743850141]
  We present a new cost-effective vision-based method that perceives humans' locations in 3D and their body orientation from a single image. We show that it is possible to rethink the concept of "social distancing" as a form of social interaction in contrast to a simple location-based rule.
  arXiv  Detail & Related papers  (2020-09-01T10:12:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.