PAWS: Perception of Articulation in the Wild at Scale from Egocentric Videos

• URL: http://arxiv.org/abs/2603.25539v1
• Date: Thu, 26 Mar 2026 15:16:51 GMT
• Title: PAWS: Perception of Articulation in the Wild at Scale from Egocentric Videos
• Abstract summary: Existing learning-based methods rely heavily on supervised training with high-quality 3D data and manual annotations.<n>We propose PAWS, a method that directly extracts object articulations from hand-object interactions in large-scale in-the-wild egocentric videos.<n>We evaluate our method on the public data sets, including HD-EPIC and Arti4D data sets, achieving significant improvements over baselines.
• Score: 90.4816165292854
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Articulation perception aims to recover the motion and structure of articulated objects (e.g., drawers and cupboards), and is fundamental to 3D scene understanding in robotics, simulation, and animation. Existing learning-based methods rely heavily on supervised training with high-quality 3D data and manual annotations, limiting scalability and diversity. To address this limitation, we propose PAWS, a method that directly extracts object articulations from hand-object interactions in large-scale in-the-wild egocentric videos. We evaluate our method on the public data sets, including HD-EPIC and Arti4D data sets, achieving significant improvements over baselines. We further demonstrate that the extracted articulations benefit downstream tasks, including fine-tuning 3D articulation prediction models and enabling robot manipulation. See the project website at https://aaltoml.github.io/PAWS/.

Related papers

• SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wild [17.556955295385908]
  Existing hand-object interaction datasets are predominantly captured in controlled studio settings.<n>We introduce a novel marker-less multi-camera system that allows for nearly unconstrained mobility in genuinely in-the-wild conditions.<n>We present SHOW3D, the first large-scale dataset with 3D annotations that show hands interacting with objects in diverse real-world environments.
  arXiv  Detail & Related papers  (2026-03-30T17:58:27Z)
• OCRA: Object-Centric Learning with 3D and Tactile Priors for Human-to-Robot Action Transfer [84.22494391514066]
  OCRA is a framework for video-based human-to-Robot Action transfer.<n>It learns directly from human demonstration videos to enable robust manipulation.
  arXiv  Detail & Related papers  (2026-03-15T14:31:03Z)
• Developing Vision-Language-Action Model from Egocentric Videos [14.1517430035289]
  Egocentric videos capture how humans manipulate objects and tools, providing diverse motion cues for learning object manipulation.<n>Prior studies that leverage such videos for training robot policies typically rely on auxiliary annotations, such as detailed hand-pose recordings.<n>In this work, we address this challenge by leveraging EgoScaler, a framework that extracts 6DoF object manipulation trajectories from egocentric videos.
  arXiv  Detail & Related papers  (2025-09-26T07:09:33Z)
• Articulated Object Estimation in the Wild [25.616481887384708]
  ArtiPoint is a novel estimation framework that can infer articulated object models under dynamic camera motion and partial observability.<n>By combining deep point tracking with a factor graph optimization framework, ArtiPoint robustly estimates articulated part trajectories and articulation axes directly from raw RGB-D videos.<n>We benchmark ArtiPoint against a range of classical and learning-based baselines, demonstrating its superior performance on Arti4D.
  arXiv  Detail & Related papers  (2025-09-01T18:34:17Z)
• Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos [66.62109400603394]
  We introduce Being-H0, a dexterous Vision-Language-Action model trained on large-scale human videos.<n>Our approach centers on physical instruction tuning, a novel training paradigm that combines large-scale VLA pretraining from human videos, physical space alignment for 3D reasoning, and post-training adaptation for robotic tasks.<n>We empirically show the excellence of Being-H0 in hand motion generation and instruction following, and it also scales well with model and data sizes.
  arXiv  Detail & Related papers  (2025-07-21T13:19:09Z)
• Object-centric 3D Motion Field for Robot Learning from Human Videos [56.9436352861611]
  We propose to use object-centric 3D motion field to represent actions for robot learning from human videos.<n>We present a novel framework for extracting this representation from videos for zero-shot control.<n> Experiments show that our method reduces 3D motion estimation error by over 50% compared to the latest method.
  arXiv  Detail & Related papers  (2025-06-04T17:59:06Z)
• SUGAR: Pre-training 3D Visual Representations for Robotics [85.55534363501131]
  We introduce a novel 3D pre-training framework for robotics named SUGAR. SUGAR captures semantic, geometric and affordance properties of objects through 3D point clouds. We show that SUGAR's 3D representation outperforms state-of-the-art 2D and 3D representations.
  arXiv  Detail & Related papers  (2024-04-01T21:23:03Z)
• HMP: Hand Motion Priors for Pose and Shape Estimation from Video [52.39020275278984]
  We develop a generative motion prior specific for hands, trained on the AMASS dataset which features diverse and high-quality hand motions. Our integration of a robust motion prior significantly enhances performance, especially in occluded scenarios. We demonstrate our method's efficacy via qualitative and quantitative evaluations on the HO3D and DexYCB datasets.
  arXiv  Detail & Related papers  (2023-12-27T22:35:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.