AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video
Anonymisation
- URL: http://arxiv.org/abs/2107.10685v2
- Date: Fri, 23 Jul 2021 16:10:18 GMT
- Title: AnonySIGN: Novel Human Appearance Synthesis for Sign Language Video
Anonymisation
- Authors: Ben Saunders, Necati Cihan Camgoz, Richard Bowden
- Abstract summary: We introduce the task of Sign Language Video Anonymisation (SLVA) as an automatic method to anonymise the visual appearance of a sign language video.
To tackle SLVA, we propose AnonySign, a novel automatic approach for visual anonymisation of sign language data.
- Score: 37.679114155300084
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The visual anonymisation of sign language data is an essential task to
address privacy concerns raised by large-scale dataset collection. Previous
anonymisation techniques have either significantly affected sign comprehension
or required manual, labour-intensive work.
In this paper, we formally introduce the task of Sign Language Video
Anonymisation (SLVA) as an automatic method to anonymise the visual appearance
of a sign language video whilst retaining the meaning of the original sign
language sequence. To tackle SLVA, we propose AnonySign, a novel automatic
approach for visual anonymisation of sign language data. We first extract pose
information from the source video to remove the original signer appearance. We
next generate a photo-realistic sign language video of a novel appearance from
the pose sequence, using image-to-image translation methods in a conditional
variational autoencoder framework. An approximate posterior style distribution
is learnt, which can be sampled from to synthesise novel human appearances. In
addition, we propose a novel \textit{style loss} that ensures style consistency
in the anonymised sign language videos.
We evaluate AnonySign for the SLVA task with extensive quantitative and
qualitative experiments highlighting both realism and anonymity of our novel
human appearance synthesis. In addition, we formalise an anonymity perceptual
study as an evaluation criteria for the SLVA task and showcase that video
anonymisation using AnonySign retains the original sign language content.
Related papers
- Pose-Based Sign Language Appearance Transfer [5.839722619084469]
We introduce a method for transferring the signer's appearance in sign language skeletal poses while preserving the sign content.
This approach improves pose-based rendering and sign stitching while obfuscating identity.
Our experiments show that while the method reduces signer identification accuracy, it slightly harms sign recognition performance.
arXiv Detail & Related papers (2024-10-17T15:33:54Z) - Anonymization Prompt Learning for Facial Privacy-Preserving Text-to-Image Generation [56.46932751058042]
We train a learnable prompt prefix for text-to-image diffusion models, which forces the model to generate anonymized facial identities.
Experiments demonstrate the successful anonymization performance of APL, which anonymizes any specific individuals without compromising the quality of non-identity-specific image generation.
arXiv Detail & Related papers (2024-05-27T07:38:26Z) - Sign Stitching: A Novel Approach to Sign Language Production [35.35777909051466]
We propose using dictionary examples to create expressive sign language sequences.
We present a 7-step approach to effectively stitch the signs together.
We leverage the SignGAN model to map the output to a photo-realistic signer.
arXiv Detail & Related papers (2024-05-13T11:44:57Z) - DiffSLVA: Harnessing Diffusion Models for Sign Language Video
Anonymization [33.18321022815901]
We introduce DiffSLVA, a novel methodology for text-guided sign language video anonymization.
We develop a specialized module dedicated to capturing facial expressions, which are critical for conveying linguistic information in signed languages.
This innovative methodology makes possible, for the first time, sign language video anonymization that could be used for real-world applications.
arXiv Detail & Related papers (2023-11-27T18:26:19Z) - Automatic dense annotation of large-vocabulary sign language videos [85.61513254261523]
We propose a simple, scalable framework to vastly increase the density of automatic annotations.
We make these annotations publicly available to support the sign language research community.
arXiv Detail & Related papers (2022-08-04T17:55:09Z) - Everybody Sign Now: Translating Spoken Language to Photo Realistic Sign
Language Video [43.45785951443149]
To be truly understandable by Deaf communities, an automatic Sign Language Production system must generate a photo-realistic signer.
We propose SignGAN, the first SLP model to produce photo-realistic continuous sign language videos directly from spoken language.
A pose-conditioned human synthesis model is then introduced to generate a photo-realistic sign language video from the skeletal pose sequence.
arXiv Detail & Related papers (2020-11-19T14:31:06Z) - Watch, read and lookup: learning to spot signs from multiple supervisors [99.50956498009094]
Given a video of an isolated sign, our task is to identify whether and where it has been signed in a continuous, co-articulated sign language video.
We train a model using multiple types of available supervision by: (1) watching existing sparsely labelled footage; (2) reading associated subtitles which provide additional weak-supervision; and (3) looking up words in visual sign language dictionaries.
These three tasks are integrated into a unified learning framework using the principles of Noise Contrastive Estimation and Multiple Instance Learning.
arXiv Detail & Related papers (2020-10-08T14:12:56Z) - Transferring Cross-domain Knowledge for Video Sign Language Recognition [103.9216648495958]
Word-level sign language recognition (WSLR) is a fundamental task in sign language interpretation.
We propose a novel method that learns domain-invariant visual concepts and fertilizes WSLR models by transferring knowledge of subtitled news sign to them.
arXiv Detail & Related papers (2020-03-08T03:05:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.