Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Poses

• URL: http://arxiv.org/abs/2012.10941v2
• Date: Mon, 4 Jan 2021 14:44:33 GMT
• Title: Can Everybody Sign Now? Exploring Sign Language Video Generation from 2D Poses
• Abstract summary: We use the state of the art in Deep Learning for motion transfer to generate videos of signers performing sign language given a 2D pose skeleton. We evaluate the generated videos quantitatively and qualitatively showing that the current models are not enough to generated adequate videos for Sign Language due to lack of detail in hands.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work have addressed the generation of human poses represented by 2D/3D coordinates of human joints for sign language. We use the state of the art in Deep Learning for motion transfer and evaluate them on How2Sign, an American Sign Language dataset, to generate videos of signers performing sign language given a 2D pose skeleton. We evaluate the generated videos quantitatively and qualitatively showing that the current models are not enough to generated adequate videos for Sign Language due to lack of detail in hands.

Related papers

• SignRefine: Adapting Foundational Video Models for Sign Language Generation [19.287012175897235]
  SignRefine is a sign language video generation model that produces comprehensible signing from 2D keypoint conditioning alone.<n>We present NVSign, a large-scale dataset of video content produced in sign language.
  arXiv  Detail & Related papers  (2026-09-08T09:38:22Z)
• DexAvatar: 3D Sign Language Reconstruction with Hand and Body Pose Priors [4.950209377730332]
  We introduce DexAvatar, a novel framework to reconstruct bio-mechanically accurate fine-grained hand articulations and body movements from sign language videos.<n>DexAvatar achieves strong performance in the SGNify motion capture dataset, reaching an improvement of 35.11% in the estimation of body and hand poses.
  arXiv  Detail & Related papers  (2025-12-24T08:44:58Z)
• SignX: The Foundation Model for Sign Recognition [28.651340554377906]
  This paper proposes SignX, a foundation model framework for sign recognition. It is a concise yet powerful framework applicable to multiple human activity recognition scenarios. Experimental results show that SignX can recognize signs from sign language video, producing predicted gloss representations with greater accuracy than has been reported in prior work.
  arXiv  Detail & Related papers  (2025-04-22T23:23:39Z)
• IlluSign: Illustrating Sign Language Videos by Leveraging the Attention Mechanism [52.64170845753814]
  This work aims to convert sign language video footage into static illustrations, which serve as an additional educational resource to complement video content. We propose a method that illustrates sign language videos by leveraging generative models' ability to understand both the semantic and geometric aspects of images. Our approach focuses on transferring a sketch like illustration style to video footage of sign language, combining the start and end frames of a sign into a single illustration.
  arXiv  Detail & Related papers  (2025-04-15T02:53:32Z)
• SignMusketeers: An Efficient Multi-Stream Approach for Sign Language Translation at Scale [22.49602248323602]
  A persistent challenge in sign language video processing is how we learn representations of sign language. Our proposed method focuses on just the most relevant parts in a signing video: the face, hands and body posture of the signer. Our approach is based on learning from individual frames (rather than video sequences) and is therefore much more efficient than prior work on sign language pre-training.
  arXiv  Detail & Related papers  (2024-06-11T03:00:41Z)
• A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars [49.60328609426056]
  Spoken2Sign is a system for translating spoken languages into sign languages. We present a simple baseline consisting of three steps: creating a gloss-video dictionary, estimating a 3D sign for each sign video, and training a Spoken2Sign model. As far as we know, we are the first to present the Spoken2Sign task in an output format of 3D signs.
  arXiv  Detail & Related papers  (2024-01-09T18:59:49Z)
• Sign Language Production with Latent Motion Transformer [2.184775414778289]
  We develop a new method to make high-quality sign videos without using human poses as a middle step. Our model works in two main parts: first, it learns from a generator and the video's hidden features, and next, it uses another model to understand the order of these hidden features. Compared with previous state-of-the-art approaches, our model performs consistently better on two word-level sign language datasets.
  arXiv  Detail & Related papers  (2023-12-20T10:53:06Z)
• DiffSLVA: Harnessing Diffusion Models for Sign Language Video Anonymization [33.18321022815901]
  We introduce DiffSLVA, a novel methodology for text-guided sign language video anonymization. We develop a specialized module dedicated to capturing facial expressions, which are critical for conveying linguistic information in signed languages. This innovative methodology makes possible, for the first time, sign language video anonymization that could be used for real-world applications.
  arXiv  Detail & Related papers  (2023-11-27T18:26:19Z)
• SignAvatars: A Large-scale 3D Sign Language Holistic Motion Dataset and Benchmark [20.11364909443987]
  SignAvatars is the first large-scale, multi-prompt 3D sign language (SL) motion dataset designed to bridge the communication gap for Deaf and hard-of-hearing individuals. The dataset comprises 70,000 videos from 153 signers, totaling 8.34 million frames, covering both isolated signs and continuous, co-articulated signs.
  arXiv  Detail & Related papers  (2023-10-31T13:15:49Z)
• Human Part-wise 3D Motion Context Learning for Sign Language Recognition [54.772707210595065]
  We propose P3D, the human part-wise motion context learning framework for sign language recognition. Our main contributions lie in two dimensions: learning the part-wise motion context and employing the pose ensemble to utilize 2D and 3D pose jointly.
  arXiv  Detail & Related papers  (2023-08-18T05:01:52Z)
• Reconstructing Signing Avatars From Video Using Linguistic Priors [54.5282429129769]
  Sign language (SL) is the primary method of communication for the 70 million Deaf people around the world. replacing video dictionaries of isolated signs with 3D avatars can aid learning and enable AR/VR applications. SGNify captures fine-grained hand pose, facial expression, and body movement fully automatically from in-the-wild monocular SL videos.
  arXiv  Detail & Related papers  (2023-04-20T17:29:50Z)
• Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
  Sign language is used by deaf or speech impaired people to communicate. Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance. Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
  arXiv  Detail & Related papers  (2021-03-16T03:38:17Z)
• Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video [43.45785951443149]
  To be truly understandable by Deaf communities, an automatic Sign Language Production system must generate a photo-realistic signer. We propose SignGAN, the first SLP model to produce photo-realistic continuous sign language videos directly from spoken language. A pose-conditioned human synthesis model is then introduced to generate a photo-realistic sign language video from the skeletal pose sequence.
  arXiv  Detail & Related papers  (2020-11-19T14:31:06Z)
• BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues [106.21067543021887]
  We show how to use mouthing cues from signers to obtain high-quality annotations from video data. The BSL-1K dataset is a collection of British Sign Language (BSL) signs of unprecedented scale.
  arXiv  Detail & Related papers  (2020-07-23T16:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.