Related papers: XmoPipe: A Pipeline for Large-Scale In-the-Wild Human Motion Dataset Construction

XmoPipe: A Pipeline for Large-Scale In-the-Wild Human Motion Dataset Construction

URL: http://arxiv.org/abs/2606.20731v1
Date: Wed, 17 Jun 2026 09:33:33 GMT
Title: XmoPipe: A Pipeline for Large-Scale In-the-Wild Human Motion Dataset Construction
Abstract summary: Large-scale human motion datasets are essential for training robust motion models for analysis, synthesis, and understanding.<n>Recent advances in monocular motion capture and video-language understanding open the way to extract plausible motion from online videos.<n>We present a scalable pipeline for constructing in-the-wild human motion datasets.
Score: 39.41824537790665
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: Large-scale human motion datasets are essential for training robust motion models for analysis, synthesis, and understanding. While marker-based motion capture provides precise data, it is costly and limited in scale and diversity. Recent advances in monocular motion capture and video-language understanding open the way to extract plausible motion from unconstrained online videos. We present a scalable pipeline for constructing in-the-wild human motion datasets. From a few keywords, the system retrieves videos, extracts 3D body and facial motion, and generates high-level textual descriptions. The pipeline is flexible, enabling targeted collection of various motions, multi-person interactions, or expressive behaviors. We demonstrate its quality by training motion reconstruction and motion generation models, showing performance comparable to models trained on traditional motion capture datasets and strong cross-dataset generalization.

Related papers

HandX: Scaling Bimanual Motion and Interaction Generation [55.486397101415]
HandX is a unified foundation spanning data, annotation, and evaluation.<n>We consolidate and filter existing datasets for quality, and collect a new motion-capture dataset.<n>Experiments demonstrate high-quality dexterous motion generation, supported by our newly proposed hand-focused metrics.
arXiv Detail & Related papers (2026-03-30T17:59:49Z)
Kimodo: Scaling Controllable Human Motion Generation [77.66868439601062]
Kimodo is a controllable kinematic motion diffusion model trained on 700 hours of optical motion capture data.<n>Our model generates high-quality motions while being easily controlled through text and a comprehensive suite of kinematic constraints.
arXiv Detail & Related papers (2026-03-16T17:09:30Z)
FoundationMotion: Auto-Labeling and Reasoning about Spatial Movement in Videos [109.99404241220039]
We introduce FoundationMotion, a fully automated data curation pipeline that constructs large-scale motion datasets.<n>Our approach first detects and tracks objects in videos to extract their trajectories, then leverages these trajectories and video frames with Large Language Models.<n>We fine-tune open-source models including NVILA-Video-15B and Qwen2.5-7B, achieving substantial improvements in motion understanding without compromising performance.
arXiv Detail & Related papers (2025-12-11T18:53:15Z)
Motion Prompting: Controlling Video Generation with Motion Trajectories [57.049252242807874]
We train a video generation model conditioned on sparse or dense video trajectories.<n>We translate high-level user requests into detailed, semi-dense motion prompts.<n>We demonstrate our approach through various applications, including camera and object motion control, "interacting" with an image, motion transfer, and image editing.
arXiv Detail & Related papers (2024-12-03T18:59:56Z)
Exploring Vision Transformers for 3D Human Motion-Language Models with Motion Patches [12.221087476416056]
We introduce "motion patches", a new representation of motion sequences, and propose using Vision Transformers (ViT) as motion encoders via transfer learning. These motion patches, created by dividing and sorting skeleton joints based on motion sequences, are robust to varying skeleton structures. We find that transfer learning with pre-trained weights of ViT obtained through training with 2D image data can boost the performance of motion analysis.
arXiv Detail & Related papers (2024-05-08T02:42:27Z)
Render In-between: Motion Guided Video Synthesis for Action Interpolation [53.43607872972194]
We propose a motion-guided frame-upsampling framework that is capable of producing realistic human motion and appearance. A novel motion model is trained to inference the non-linear skeletal motion between frames by leveraging a large-scale motion-capture dataset. Our pipeline only requires low-frame-rate videos and unpaired human motion data but does not require high-frame-rate videos for training.
arXiv Detail & Related papers (2021-11-01T15:32:51Z)
Recognition and Synthesis of Object Transport Motion [0.0]
This project illustrates how deep convolutional networks can be used, alongside specialized data augmentation techniques, on a small motion capture dataset. The project shows how these same augmentation techniques can be scaled up for use in the more complex task of motion synthesis. By exploring recent developments in the concept of Generative Adversarial Models (GANs), specifically the Wasserstein GAN, this project outlines a model that is able to successfully generate lifelike object transportation motions.
arXiv Detail & Related papers (2020-09-27T22:13:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.