Multimodal Active Measurement for Human Mesh Recovery in Close Proximity

• URL: http://arxiv.org/abs/2310.08116v5
• Date: Tue, 08 Oct 2024 19:54:08 GMT
• Title: Multimodal Active Measurement for Human Mesh Recovery in Close Proximity
• Authors: Takahiro Maeda, Keisuke Takeshita, Norimichi Ukita, Kazuhito Tanaka,
• Abstract summary: In physical human-robot interactions, a robot needs to estimate the accurate body pose of a target person. In these pHRI scenarios, the robot cannot fully observe the target person's body with equipped cameras because the target person must be close to the robot for physical interaction. We propose an active measurement and sensor fusion framework of the equipped cameras with touch and ranging sensors such as 2D LiDAR.
• Score: 13.265259738826302
• License:
• Abstract: For physical human-robot interactions (pHRI), a robot needs to estimate the accurate body pose of a target person. However, in these pHRI scenarios, the robot cannot fully observe the target person's body with equipped cameras because the target person must be close to the robot for physical interaction. This close distance leads to severe truncation and occlusions and thus results in poor accuracy of human pose estimation. For better accuracy in this challenging environment, we propose an active measurement and sensor fusion framework of the equipped cameras with touch and ranging sensors such as 2D LiDAR. Touch and ranging sensor measurements are sparse but reliable and informative cues for localizing human body parts. In our active measurement process, camera viewpoints and sensor placements are dynamically optimized to measure body parts with higher estimation uncertainty, which is closely related to truncation or occlusion. In our sensor fusion process, assuming that the measurements of touch and ranging sensors are more reliable than the camera-based estimations, we fuse the sensor measurements to the camera-based estimated pose by aligning the estimated pose towards the measured points. Our proposed method outperformed previous methods on the standard occlusion benchmark with simulated active measurement. Furthermore, our method reliably estimated human poses using a real robot, even with practical constraints such as occlusion by blankets.

Related papers

• Fusing uncalibrated IMUs and handheld smartphone video to reconstruct knee kinematics [1.5728609542259502]
  We present a method to combine handheld smartphone video and uncalibrated wearable sensor data at their full temporal resolution. We validate this in a mixture of people with no gait impairments, lower limb prosthesis users, and individuals with a history of stroke.
  arXiv  Detail & Related papers  (2024-05-27T17:23:16Z)
• Exploring 3D Human Pose Estimation and Forecasting from the Robot's Perspective: The HARPER Dataset [52.22758311559]
  We introduce HARPER, a novel dataset for 3D body pose estimation and forecast in dyadic interactions between users and Spot. The key-novelty is the focus on the robot's perspective, i.e., on the data captured by the robot's sensors. The scenario underlying HARPER includes 15 actions, of which 10 involve physical contact between the robot and users.
  arXiv  Detail & Related papers  (2024-03-21T14:53:50Z)
• DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion [10.439802168557513]
  Motion capture from a limited number of body-worn sensors has important applications in health, human performance, and entertainment. Recent work has focused on accurately reconstructing whole-body motion from a specific sensor configuration using six IMUs. We propose a single diffusion model, DiffusionPoser, which reconstructs human motion in real-time from an arbitrary combination of sensors.
  arXiv  Detail & Related papers  (2023-08-31T12:36:50Z)
• Design Space Exploration on Efficient and Accurate Human Pose Estimation from Sparse IMU-Sensing [0.04594153909580514]
  Human Pose Estimation (HPE) to assess human motion in sports, rehabilitation or work safety requires accurate sensing without compromising personal data. Central trade-off between accuracy and efficient use of hardware resources is rarely discussed in research. We generate IMU-data from a publicly available body model dataset for different sensor configurations and train a deep learning model with this data.
  arXiv  Detail & Related papers  (2023-07-21T13:36:49Z)
• UltraGlove: Hand Pose Estimation with Mems-Ultrasonic Sensors [14.257535961674021]
  We propose a novel and low-cost hand-tracking glove that utilizes several MEMS-ultrasonic sensors attached to the fingers. Our experimental results demonstrate that this approach is both accurate, size-agnostic, and robust to external interference.
  arXiv  Detail & Related papers  (2023-06-22T03:41:47Z)
• Extrinsic Camera Calibration with Semantic Segmentation [60.330549990863624]
  We present an extrinsic camera calibration approach that automatizes the parameter estimation by utilizing semantic segmentation information. Our approach relies on a coarse initial measurement of the camera pose and builds on lidar sensors mounted on a vehicle. We evaluate our method on simulated and real-world data to demonstrate low error measurements in the calibration results.
  arXiv  Detail & Related papers  (2022-08-08T07:25:03Z)
• Single View Physical Distance Estimation using Human Pose [18.9877515094788]
  We propose a fully automated system that simultaneously estimates the camera intrinsics, the ground plane, and physical distances between people from a single RGB image or video. The proposed approach enables existing camera systems to measure physical distances without needing a dedicated calibration process or range sensors. We contribute to the publicly available MEVA dataset with additional distance annotations, resulting in MEVADA -- the first evaluation benchmark in the world for the pose-based auto-calibration and distance estimation problem.
  arXiv  Detail & Related papers  (2021-06-18T19:50:40Z)
• Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors [71.29186299435423]
  We introduce (HPS) Human POSEitioning System, a method to recover the full 3D pose of a human registered with a 3D scan of the surrounding environment. We show that our optimization-based integration exploits the benefits of the two, resulting in pose accuracy free of drift. HPS could be used for VR/AR applications where humans interact with the scene without requiring direct line of sight with an external camera.
  arXiv  Detail & Related papers  (2021-03-31T17:58:31Z)
• Automatic Social Distance Estimation From Images: Performance Evaluation, Test Benchmark, and Algorithm [78.88882860340797]
  COVID-19 virus has caused a global pandemic since March 2020. Maintaining a minimum of one meter distance from other people is strongly suggested to reduce the risk of infection. There is no suitable test benchmark for such algorithms.
  arXiv  Detail & Related papers  (2021-03-11T16:15:20Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• Learning Camera Miscalibration Detection [83.38916296044394]
  This paper focuses on a data-driven approach to learn the detection of miscalibration in vision sensors, specifically RGB cameras. Our contributions include a proposed miscalibration metric for RGB cameras and a novel semi-synthetic dataset generation pipeline based on this metric. By training a deep convolutional neural network, we demonstrate the effectiveness of our pipeline to identify whether a recalibration of the camera's intrinsic parameters is required or not.
  arXiv  Detail & Related papers  (2020-05-24T10:32:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.