SynthoGestures: A Novel Framework for Synthetic Dynamic Hand Gesture
Generation for Driving Scenarios
- URL: http://arxiv.org/abs/2309.04421v1
- Date: Fri, 8 Sep 2023 16:32:56 GMT
- Title: SynthoGestures: A Novel Framework for Synthetic Dynamic Hand Gesture
Generation for Driving Scenarios
- Authors: Amr Gomaa and Robin Zitt and Guillermo Reyes and Antonio Kr\"uger
- Abstract summary: Our framework synthesizes realistic hand gestures, offering customization options and reducing the risk of overfitting.
We simulate different camera locations and types, such as RGB, infrared, and depth cameras, without incurring additional time and cost.
By saving time and effort in the creation of the data set, our tool accelerates the development of gesture recognition systems for automotive applications.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Creating a diverse and comprehensive dataset of hand gestures for dynamic
human-machine interfaces in the automotive domain can be challenging and
time-consuming. To overcome this challenge, we propose using synthetic gesture
datasets generated by virtual 3D models. Our framework utilizes Unreal Engine
to synthesize realistic hand gestures, offering customization options and
reducing the risk of overfitting. Multiple variants, including gesture speed,
performance, and hand shape, are generated to improve generalizability. In
addition, we simulate different camera locations and types, such as RGB,
infrared, and depth cameras, without incurring additional time and cost to
obtain these cameras. Experimental results demonstrate that our proposed
framework,
SynthoGestures\footnote{\url{https://github.com/amrgomaaelhady/SynthoGestures}},
improves gesture recognition accuracy and can replace or augment real-hand
datasets. By saving time and effort in the creation of the data set, our tool
accelerates the development of gesture recognition systems for automotive
applications.
Related papers
- HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions [68.28684509445529]
We present HandBooster, a new approach to uplift the data diversity and boost the 3D hand-mesh reconstruction performance.
First, we construct versatile content-aware conditions to guide a diffusion model to produce realistic images with diverse hand appearances, poses, views, and backgrounds.
Then, we design a novel condition creator based on our similarity-aware distribution sampling strategies to deliberately find novel and realistic interaction poses that are distinctive from the training set.
arXiv Detail & Related papers (2024-03-27T13:56:08Z) - Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
TRUMANS is the most comprehensive motion-captured HSI dataset currently available.
It intricately captures whole-body human motions and part-level object dynamics.
We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
arXiv Detail & Related papers (2024-03-13T15:45:04Z) - GTAutoAct: An Automatic Datasets Generation Framework Based on Game
Engine Redevelopment for Action Recognition [12.521014978532548]
GTAutoAct is a novel dataset generation framework leveraging game engine technology to facilitate advancements in action recognition.
It transforms coordinate-based 3D human motion into rotation-orientated representation with enhanced suitability in multiple viewpoints.
It implements an autonomous video capture and processing pipeline, featuring a randomly navigating camera, with auto-trimming and labeling functionalities.
arXiv Detail & Related papers (2024-01-24T12:18:31Z) - MuSHRoom: Multi-Sensor Hybrid Room Dataset for Joint 3D Reconstruction and Novel View Synthesis [26.710960922302124]
We propose a real-world Multi-Sensor Hybrid Room dataset (MuSHRoom)
Our dataset presents exciting challenges and requires state-of-the-art methods to be cost-effective, robust to noisy data and devices.
We benchmark several famous pipelines on our dataset for joint 3D mesh reconstruction and novel view synthesis.
arXiv Detail & Related papers (2023-11-05T21:46:12Z) - Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps [100.72245315180433]
We present a reconfigurable data glove design to capture different modes of human hand-object interactions.
The glove operates in three modes for various downstream tasks with distinct features.
We evaluate the system's three modes by (i) recording hand gestures and associated forces, (ii) improving manipulation fluency in VR, and (iii) producing realistic simulation effects of various tool uses.
arXiv Detail & Related papers (2023-01-14T05:35:50Z) - MoDi: Unconditional Motion Synthesis from Diverse Data [51.676055380546494]
We present MoDi, an unconditional generative model that synthesizes diverse motions.
Our model is trained in a completely unsupervised setting from a diverse, unstructured and unlabeled motion dataset.
We show that despite the lack of any structure in the dataset, the latent space can be semantically clustered.
arXiv Detail & Related papers (2022-06-16T09:06:25Z) - Hands-Up: Leveraging Synthetic Data for Hands-On-Wheel Detection [0.38233569758620045]
This work demonstrates the use of synthetic photo-realistic in-cabin data to train a Driver Monitoring System.
We show how performing error analysis and generating the missing edge-cases in our platform boosts performance.
This showcases the ability of human-centric synthetic data to generalize well to the real world.
arXiv Detail & Related papers (2022-05-31T23:34:12Z) - Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes.
We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature.
We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
arXiv Detail & Related papers (2022-04-22T03:17:35Z) - RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB
Video [76.86512780916827]
We present the first real-time method for motion capture of skeletal pose and 3D surface geometry of hands from a single RGB camera.
In order to address the inherent depth ambiguities in RGB data, we propose a novel multi-task CNN.
We experimentally verify the individual components of our RGB two-hand tracking and 3D reconstruction pipeline.
arXiv Detail & Related papers (2021-06-22T12:53:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.