ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate

• URL: http://arxiv.org/abs/2503.21268v1
• Date: Thu, 27 Mar 2025 08:49:33 GMT
• Title: ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinate
• Authors: Ming Yan, Xincheng Lin, Yuhua Luo, Shuqi Fan, Yudi Dai, Qixin Zhong, Lincai Zhong, Yuexin Ma, Lan Xu, Chenglu Wen, Siqi Shen, Cheng Wang,
• Abstract summary: Human Motion Recovery (HMR) research mainly focuses on ground-based motions such as running.<n>To address the insufficiency of climbing motion datasets, we collect AscendMotion, a large-scale well-annotated, and challenging climbing motion dataset.<n>We propose ClimbingCap, a motion recovery method that reconstructs continuous 3D human climbing motion in a global coordinate system.
• Score: 41.90481583757255
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Human Motion Recovery (HMR) research mainly focuses on ground-based motions such as running. The study on capturing climbing motion, an off-ground motion, is sparse. This is partly due to the limited availability of climbing motion datasets, especially large-scale and challenging 3D labeled datasets. To address the insufficiency of climbing motion datasets, we collect AscendMotion, a large-scale well-annotated, and challenging climbing motion dataset. It consists of 412k RGB, LiDAR frames, and IMU measurements, including the challenging climbing motions of 22 skilled climbing coaches across 12 different rock walls. Capturing the climbing motions is challenging as it requires precise recovery of not only the complex pose but also the global position of climbers. Although multiple global HMR methods have been proposed, they cannot faithfully capture climbing motions. To address the limitations of HMR methods for climbing, we propose ClimbingCap, a motion recovery method that reconstructs continuous 3D human climbing motion in a global coordinate system. One key insight is to use the RGB and LiDAR modalities to separately reconstruct motions in camera coordinates and global coordinates and to optimize them jointly. We demonstrate the quality of the AscendMotion dataset and present promising results from ClimbingCap. The AscendMotion dataset and source code release publicly at \href{this link}{http://www.lidarhumanmotion.net/climbingcap/}

Related papers

• Mocap-2-to-3: Multi-view Lifting for Monocular Motion Recovery with 2D Pretraining [49.223455189395025]
  Mocap-2-to-3 is a novel framework that performs multi-view lifting from monocular input.<n>To leverage abundant 2D data, we decompose complex 3D motion into multi-view syntheses.<n>Our method surpasses state-of-the-art approaches in both camera-space motion realism and world-grounded human positioning.
  arXiv  Detail & Related papers  (2025-03-05T06:32:49Z)
• World-Grounded Human Motion Recovery via Gravity-View Coordinates [60.618543026949226]
  We propose estimating human poses in a novel Gravity-View coordinate system. The proposed GV system is naturally gravity-aligned and uniquely defined for each video frame. Our method recovers more realistic motion in both the camera space and world-grounded settings, outperforming state-of-the-art methods in both accuracy and speed.
  arXiv  Detail & Related papers  (2024-09-10T17:25:47Z)
• WANDR: Intention-guided Human Motion Generation [67.07028110459787]
  We introduce WANDR, a data-driven model that takes an avatar's initial pose and a goal's 3D position and generates natural human motions that place the end effector (wrist) on the goal location. Intention guides the agent to the goal, and interactively adapts the generation to novel situations without needing to define sub-goals or the entire motion path. We evaluate our method extensively and demonstrate its ability to generate natural and long-term motions that reach 3D goals and to unseen goal locations.
  arXiv  Detail & Related papers  (2024-04-23T10:20:17Z)
• CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactions [41.593477269767924]
  We collect CIMI4D, a large rock textbfCltextbfImbing textbfMottextbfIon dataset from 12 persons climbing 13 different climbing walls. The dataset consists of around 180,000 frames of pose inertial measurements, LiDAR point clouds, RGB videos, high-precision static point cloud scenes, and reconstructed scene meshes.
  arXiv  Detail & Related papers  (2023-03-31T10:26:47Z)
• Decoupling Human and Camera Motion from Videos in the Wild [67.39432972193929]
  We propose a method to reconstruct global human trajectories from videos in the wild. Our method decouples the camera and human motion, which allows us to place people in the same world coordinate frame.
  arXiv  Detail & Related papers  (2023-02-24T18:59:15Z)
• NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action [24.67958500694608]
  We introduce the Neural Motion (NeMo) field to represent the underlying 3D motions across a set of videos of the same action. NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection.
  arXiv  Detail & Related papers  (2022-12-28T01:40:32Z)
• HSC4D: Human-centered 4D Scene Capture in Large-scale Indoor-outdoor Space Using Wearable IMUs and LiDAR [51.9200422793806]
  Using only body-mounted IMUs and LiDAR, HSC4D is space-free without any external devices' constraints and map-free without pre-built maps. Relationships between humans and environments are also explored to make their interaction more realistic.
  arXiv  Detail & Related papers  (2022-03-17T10:05:55Z)
• augKlimb: Interactive Data-Led Augmentation of Bouldering Training [0.0]
  Climbing is a popular sport, especially indoors, where climbers can train on man-made routes using artificial holds. Various aspects of adding computer-interaction to climbing have been studied in recent years. There is a large space for research into lightweight tools to aid recreational intermediate climbers.
  arXiv  Detail & Related papers  (2020-01-22T10:26:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.