Self-supervised Optimization of Hand Pose Estimation using Anatomical
Features and Iterative Learning
- URL: http://arxiv.org/abs/2307.03007v1
- Date: Thu, 6 Jul 2023 14:13:11 GMT
- Title: Self-supervised Optimization of Hand Pose Estimation using Anatomical
Features and Iterative Learning
- Authors: Christian Jauch, Timo Leitritz, Marco F. Huber
- Abstract summary: This paper presents a self-supervised pipeline for adapting hand pose estimation to specific use cases with minimal human interaction.
The pipeline consists of a general machine learning model for hand pose estimation trained on a generalized dataset.
The effectiveness of the pipeline is demonstrated by training an activity recognition as a downstream task in the manual assembly scenario.
- Score: 4.698846136465861
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Manual assembly workers face increasing complexity in their work.
Human-centered assistance systems could help, but object recognition as an
enabling technology hinders sophisticated human-centered design of these
systems. At the same time, activity recognition based on hand poses suffers
from poor pose estimation in complex usage scenarios, such as wearing gloves.
This paper presents a self-supervised pipeline for adapting hand pose
estimation to specific use cases with minimal human interaction. This enables
cheap and robust hand posebased activity recognition. The pipeline consists of
a general machine learning model for hand pose estimation trained on a
generalized dataset, spatial and temporal filtering to account for anatomical
constraints of the hand, and a retraining step to improve the model. Different
parameter combinations are evaluated on a publicly available and annotated
dataset. The best parameter and model combination is then applied to unlabelled
videos from a manual assembly scenario. The effectiveness of the pipeline is
demonstrated by training an activity recognition as a downstream task in the
manual assembly scenario.
Related papers
- Fast and Expressive Gesture Recognition using a Combination-Homomorphic
Electromyogram Encoder [21.25126610043744]
We study the task of gesture recognition from electromyography (EMG)
We define combination gestures consisting of a direction component and a modifier component.
New subjects only demonstrate the single component gestures.
We extrapolate to unseen combination gestures by combining the feature vectors of real single gestures to produce synthetic training data.
arXiv Detail & Related papers (2023-10-30T20:03:34Z) - Intention estimation from gaze and motion features for human-robot
shared-control object manipulation [1.128708201885454]
Shared control can help in teleoperated object manipulation by assisting with the execution of the user's intention.
An intention estimation framework is presented, which uses natural gaze and motion features to predict the current action and the target object.
arXiv Detail & Related papers (2022-08-18T07:53:19Z) - 3D Interacting Hand Pose Estimation by Hand De-occlusion and Removal [85.30756038989057]
Estimating 3D interacting hand pose from a single RGB image is essential for understanding human actions.
We propose to decompose the challenging interacting hand pose estimation task and estimate the pose of each hand separately.
Experiments show that the proposed method significantly outperforms previous state-of-the-art interacting hand pose estimation approaches.
arXiv Detail & Related papers (2022-07-22T13:04:06Z) - Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared
Control on the Hannes Prosthesis [6.517935794312337]
We present an eye-in-hand learning-based approach for hand pre-shape classification from RGB sequences.
We tackle the peculiarity of the eye-in-hand setting by means of a model for the human arm trajectories.
arXiv Detail & Related papers (2022-03-18T09:16:48Z) - Towards unconstrained joint hand-object reconstruction from RGB videos [81.97694449736414]
Reconstructing hand-object manipulations holds a great potential for robotics and learning from human demonstrations.
We first propose a learning-free fitting approach for hand-object reconstruction which can seamlessly handle two-hand object interactions.
arXiv Detail & Related papers (2021-08-16T12:26:34Z) - Real-time Pose and Shape Reconstruction of Two Interacting Hands With a
Single Depth Camera [79.41374930171469]
We present a novel method for real-time pose and shape reconstruction of two strongly interacting hands.
Our approach combines an extensive list of favorable properties, namely it is marker-less.
We show state-of-the-art results in scenes that exceed the complexity level demonstrated by previous work.
arXiv Detail & Related papers (2021-06-15T11:39:49Z) - Adversarial Motion Modelling helps Semi-supervised Hand Pose Estimation [116.07661813869196]
We propose to combine ideas from adversarial training and motion modelling to tap into unlabeled videos.
We show that an adversarial leads to better properties of the hand pose estimator via semi-supervised training on unlabeled video sequences.
The main advantage of our approach is that we can make use of unpaired videos and joint sequence data both of which are much easier to attain than paired training data.
arXiv Detail & Related papers (2021-06-10T17:50:19Z) - Locally Aware Piecewise Transformation Fields for 3D Human Mesh
Registration [67.69257782645789]
We propose piecewise transformation fields that learn 3D translation vectors to map any query point in posed space to its correspond position in rest-pose space.
We show that fitting parametric models with poses by our network results in much better registration quality, especially for extreme poses.
arXiv Detail & Related papers (2021-04-16T15:16:09Z) - Temporally Guided Articulated Hand Pose Tracking in Surgical Videos [27.525545343598527]
Articulated hand pose tracking is an underexplored problem that carries the potential for use in an extensive number of applications.
We propose a novel hand pose estimation model, Res152- CondPose, which improves tracking accuracy by incorporating a hand pose prior to its pose prediction.
Our dataset contains 76 video clips from 28 publicly available surgical videos and over 8.1k annotated hand pose instances.
arXiv Detail & Related papers (2021-01-12T03:44:04Z) - Measuring Generalisation to Unseen Viewpoints, Articulations, Shapes and
Objects for 3D Hand Pose Estimation under Hand-Object Interaction [137.28465645405655]
HANDS'19 is a challenge to evaluate the abilities of current 3D hand pose estimators (HPEs) to interpolate and extrapolate the poses of a training set.
We show that the accuracy of state-of-the-art methods can drop, and that they fail mostly on poses absent from the training set.
arXiv Detail & Related papers (2020-03-30T19:28:13Z) - Nonparametric Structure Regularization Machine for 2D Hand Pose
Estimation [21.250031729596085]
Hand pose estimation is more challenging than body pose estimation due to severe articulation, self-occlusion and high dexterity of the hand.
We propose a novel Nonparametric Structure Regularization Machine (NSRM) for 2D hand pose estimation, adopting a cascade multi-task architecture to learn hand structure and keypoint representations jointly.
arXiv Detail & Related papers (2020-01-24T03:27:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.