AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisation

• URL: http://arxiv.org/abs/2107.10685v2
• Date: Fri, 23 Jul 2021 16:10:18 GMT
• Title: AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video Anonymisation
• Authors: Ben Saunders, Necati Cihan Camgoz, Richard Bowden
• Abstract summary: We introduce the task of Sign Language Video Anonymisation (SLVA) as an automatic method to anonymise the visual appearance of a sign language video. To tackle SLVA, we propose AnonySign, a novel automatic approach for visual anonymisation of sign language data.
• Score: 37.679114155300084
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The visual anonymisation of sign language data is an essential task to address privacy concerns raised by large-scale dataset collection. Previous anonymisation techniques have either significantly affected sign comprehension or required manual, labour-intensive work. In this paper, we formally introduce the task of Sign Language Video Anonymisation (SLVA) as an automatic method to anonymise the visual appearance of a sign language video whilst retaining the meaning of the original sign language sequence. To tackle SLVA, we propose AnonySign, a novel automatic approach for visual anonymisation of sign language data. We first extract pose information from the source video to remove the original signer appearance. We next generate a photo-realistic sign language video of a novel appearance from the pose sequence, using image-to-image translation methods in a conditional variational autoencoder framework. An approximate posterior style distribution is learnt, which can be sampled from to synthesise novel human appearances. In addition, we propose a novel \textit{style loss} that ensures style consistency in the anonymised sign language videos. We evaluate AnonySign for the SLVA task with extensive quantitative and qualitative experiments highlighting both realism and anonymity of our novel human appearance synthesis. In addition, we formalise an anonymity perceptual study as an evaluation criteria for the SLVA task and showcase that video anonymisation using AnonySign retains the original sign language content.

Related papers

• DiffSign: AI-Assisted Generation of Customizable Sign Language Videos With Enhanced Realism [1.6536018920603175]
  We create sign language videos with synthetic signers that are realistic and expressive. Our approach combines parametric modeling and generative modeling to generate realistic-looking synthetic signers. Our results show that the sign language videos generated using our approach have better temporal consistency and realism than signing videos generated by a diffusion model conditioned only on text prompts.
  arXiv  Detail & Related papers  (2024-12-05T05:18:28Z)
• Signs as Tokens: An Autoregressive Multilingual Sign Language Generator [55.94334001112357]
  We introduce a multilingual sign language model, Signs as Tokens (SOKE), which can generate 3D sign avatars autoregressively from text inputs. To align sign language with the LM, we develop a decoupled tokenizer that discretizes continuous signs into token sequences representing various body parts. These sign tokens are integrated into the raw text vocabulary of the LM, allowing for supervised fine-tuning on sign language datasets.
  arXiv  Detail & Related papers  (2024-11-26T18:28:09Z)
• Pose-Based Sign Language Appearance Transfer [5.839722619084469]
  We introduce a method for transferring the signer's appearance in sign language skeletal poses while preserving the sign content. This approach improves pose-based rendering and sign stitching while obfuscating identity. Our experiments show that while the method reduces signer identification accuracy, it slightly harms sign recognition performance.
  arXiv  Detail & Related papers  (2024-10-17T15:33:54Z)
• Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation [56.46932751058042]
  We train a learnable prompt prefix for text-to-image diffusion models, which forces the model to generate anonymized facial identities. Experiments demonstrate the successful anonymization performance of APL, which anonymizes any specific individuals without compromising the quality of non-identity-specific image generation.
  arXiv  Detail & Related papers  (2024-05-27T07:38:26Z)
• DiffSLVA: Harnessing Diffusion Models for Sign Language Video Anonymization [33.18321022815901]
  We introduce DiffSLVA, a novel methodology for text-guided sign language video anonymization. We develop a specialized module dedicated to capturing facial expressions, which are critical for conveying linguistic information in signed languages. This innovative methodology makes possible, for the first time, sign language video anonymization that could be used for real-world applications.
  arXiv  Detail & Related papers  (2023-11-27T18:26:19Z)
• Automatic dense annotation of large-vocabulary sign language videos [85.61513254261523]
  We propose a simple, scalable framework to vastly increase the density of automatic annotations. We make these annotations publicly available to support the sign language research community.
  arXiv  Detail & Related papers  (2022-08-04T17:55:09Z)
• Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign Language Video [43.45785951443149]
  To be truly understandable by Deaf communities, an automatic Sign Language Production system must generate a photo-realistic signer. We propose SignGAN, the first SLP model to produce photo-realistic continuous sign language videos directly from spoken language. A pose-conditioned human synthesis model is then introduced to generate a photo-realistic sign language video from the skeletal pose sequence.
  arXiv  Detail & Related papers  (2020-11-19T14:31:06Z)
• Transferring Cross-domain Knowledge for Video Sign Language Recognition [103.9216648495958]
  Word-level sign language recognition (WSLR) is a fundamental task in sign language interpretation. We propose a novel method that learns domain-invariant visual concepts and fertilizes WSLR models by transferring knowledge of subtitled news sign to them.
  arXiv  Detail & Related papers  (2020-03-08T03:05:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.