Open Source HamNoSys Parser for Multilingual Sign Language Encoding
- URL: http://arxiv.org/abs/2204.06924v1
- Date: Thu, 14 Apr 2022 12:33:33 GMT
- Title: Open Source HamNoSys Parser for Multilingual Sign Language Encoding
- Authors: Sylwia Majchrowska and Marta Plantykow and Milena Olech
- Abstract summary: This paper presents an automated tool to convert HamNoSys annotations into numerical labels.
Our proposed numerical multilabels greatly simplify the structure of HamNoSys annotation without significant loss of gloss meaning.
These numerical multilabels can potentially be used to feed the machine learning models, which would accelerate the development of vision-based sign language recognition.
- Score: 3.867363075280544
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: This paper presents our recent developments in the field of automatic
processing of sign language corpora using the Hamburg Sign Language Annotation
System (HamNoSys). We designed an automated tool to convert HamNoSys
annotations into numerical labels for defined initial features of body and hand
positions. Our proposed numerical multilabels greatly simplify the structure of
HamNoSys annotation without significant loss of gloss meaning. These numerical
multilabels can potentially be used to feed the machine learning models, which
would accelerate the development of vision-based sign language recognition. In
addition, this tool can assist experts in the annotation process to help
identify semantic errors. The code and sample annotations are publicly
available at https://github.com/hearai/parse-hamnosys.
Related papers
- FASSILA: A Corpus for Algerian Dialect Fake News Detection and Sentiment Analysis [0.0]
The Algerian dialect (AD) faces challenges due to the absence of annotated corpora.
This study outlines the development process of a specialized corpus for Fake News (FN) detection and sentiment analysis (SA) in AD called FASSILA.
arXiv Detail & Related papers (2024-11-07T10:39:10Z) - Cross-Attention Based Influence Model for Manual and Nonmanual Sign Language Analysis [4.815056447245669]
Both manual (relating to the use of hands) and non-manual markers (NMM), such as facial expressions or mouthing cues, are important for providing the complete meaning of phrases in American Sign Language (ASL)
We present a sign language translation architecture consisting of two-stream encoders, with one encoder handling the face and the other handling the upper body (with hands)
We propose a new parallel cross-attention decoding mechanism that is useful for quantifying the influence of each input modality on the output.
arXiv Detail & Related papers (2024-09-12T15:55:39Z) - T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text [59.57676466961787]
We propose a novel dynamic vector quantization (DVA-VAE) model that can adjust the encoding length based on the information density in sign language.
Experiments conducted on the PHOENIX14T dataset demonstrate the effectiveness of our proposed method.
We propose a new large German sign language dataset, PHOENIX-News, which contains 486 hours of sign language videos, audio, and transcription texts.
arXiv Detail & Related papers (2024-06-11T10:06:53Z) - On the Importance of Sign Labeling: The Hamburg Sign Language Notation
System Case Study [5.849513679510834]
We analyze the HamNoSys labels provided by various maintainers of open sign language corpora in five sign languages.
Our findings provide valuable insights into the limitations of the current labeling methods.
arXiv Detail & Related papers (2023-01-19T11:11:57Z) - SciAnnotate: A Tool for Integrating Weak Labeling Sources for Sequence
Labeling [55.71459234749639]
SciAnnotate is a web-based tool for text annotation called SciAnnotate, which stands for scientific annotation tool.
Our tool provides users with multiple user-friendly interfaces for creating weak labels.
In this study, we take multi-source weak label denoising as an example, we utilized a Bertifying Conditional Hidden Markov Model to denoise the weak label generated by our tool.
arXiv Detail & Related papers (2022-08-07T19:18:13Z) - Automatic dense annotation of large-vocabulary sign language videos [85.61513254261523]
We propose a simple, scalable framework to vastly increase the density of automatic annotations.
We make these annotations publicly available to support the sign language research community.
arXiv Detail & Related papers (2022-08-04T17:55:09Z) - Scaling up sign spotting through sign language dictionaries [99.50956498009094]
The focus of this work is $textitsign spotting$ - given a video of an isolated sign, our task is to identify $textitwhether$ and $textitwhere$ it has been signed in a continuous, co-articulated sign language video.
We train a model using multiple types of available supervision by: (1) $textitwatching$ existing footage which is sparsely labelled using mouthing cues; (2) $textitreading$ associated subtitles which provide additional translations of the signed content.
We validate the effectiveness of our approach on low
arXiv Detail & Related papers (2022-05-09T10:00:03Z) - Label Semantics for Few Shot Named Entity Recognition [68.01364012546402]
We study the problem of few shot learning for named entity recognition.
We leverage the semantic information in the names of the labels as a way of giving the model additional signal and enriched priors.
Our model learns to match the representations of named entities computed by the first encoder with label representations computed by the second encoder.
arXiv Detail & Related papers (2022-03-16T23:21:05Z) - Watch, read and lookup: learning to spot signs from multiple supervisors [99.50956498009094]
Given a video of an isolated sign, our task is to identify whether and where it has been signed in a continuous, co-articulated sign language video.
We train a model using multiple types of available supervision by: (1) watching existing sparsely labelled footage; (2) reading associated subtitles which provide additional weak-supervision; and (3) looking up words in visual sign language dictionaries.
These three tasks are integrated into a unified learning framework using the principles of Noise Contrastive Estimation and Multiple Instance Learning.
arXiv Detail & Related papers (2020-10-08T14:12:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.