ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition

• URL: http://arxiv.org/abs/2304.05934v2
• Date: Tue, 20 Jun 2023 03:20:18 GMT
• Title: ASL Citizen: A Community-Sourced Dataset for Advancing Isolated Sign Language Recognition
• Authors: Aashaka Desai, Lauren Berger, Fyodor O. Minakov, Vanessa Milan, Chinmay Singh, Kriston Pumphrey, Richard E. Ladner, Hal Daum\'e III, Alex X. Lu, Naomi Caselli, Danielle Bragg
• Abstract summary: Sign languages are used as a primary language by approximately 70 million D/deaf people world-wide. To help tackle this problem, we release ASL Citizen, the first crowdsourced Isolated Sign Language Recognition dataset. We propose that this dataset be used for sign language dictionary retrieval for American Sign Language (ASL), where a user demonstrates a sign to their webcam to retrieve matching signs from a dictionary.
• Score: 6.296362537531586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sign languages are used as a primary language by approximately 70 million D/deaf people world-wide. However, most communication technologies operate in spoken and written languages, creating inequities in access. To help tackle this problem, we release ASL Citizen, the first crowdsourced Isolated Sign Language Recognition (ISLR) dataset, collected with consent and containing 83,399 videos for 2,731 distinct signs filmed by 52 signers in a variety of environments. We propose that this dataset be used for sign language dictionary retrieval for American Sign Language (ASL), where a user demonstrates a sign to their webcam to retrieve matching signs from a dictionary. We show that training supervised machine learning classifiers with our dataset advances the state-of-the-art on metrics relevant for dictionary retrieval, achieving 63% accuracy and a recall-at-10 of 91%, evaluated entirely on videos of users who are not present in the training or validation sets. An accessible PDF of this article is available at the following link: https://aashakadesai.github.io/research/ASLCitizen_arxiv_updated.pdf

Related papers

• SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction [65.1590372072555]
  We introduce SHuBERT, a self-supervised transformer encoder that learns strong representations from American Sign Language (ASL) video content. Inspired by the success of the HuBERT speech representation model, SHuBERT adapts masked prediction for multi-stream visual sign language input. SHuBERT achieves state-of-the-art performance across multiple benchmarks.
  arXiv  Detail & Related papers  (2024-11-25T03:13:08Z)
• Bukva: Russian Sign Language Alphabet [75.42794328290088]
  This paper investigates the recognition of the Russian fingerspelling alphabet, also known as the Russian Sign Language (RSL) dactyl. Dactyl is a component of sign languages where distinct hand movements represent individual letters of a written language. We provide Bukva, the first full-fledged open-source video dataset for RSL dactyl recognition.
  arXiv  Detail & Related papers  (2024-10-11T09:59:48Z)
• iSign: A Benchmark for Indian Sign Language Processing [5.967764101493575]
  iSign is a benchmark for Indian Sign Language (ISL) processing. We release one of the largest ISL-English datasets with more than 118K video-sentence/phrase pairs. We provide insights into the proposed benchmarks with a few linguistic insights into the workings of ISL.
  arXiv  Detail & Related papers  (2024-07-07T15:07:35Z)
• LSA64: An Argentinian Sign Language Dataset [42.27617228521691]
  This paper presents a dataset of 64 signs from the Argentinian Sign Language (LSA) The dataset, called LSA64, contains 3200 videos of 64 different LSA signs recorded by 10 subjects. We also present a pre-processed version of the dataset, from which we computed statistics of movement, position and handshape of the signs.
  arXiv  Detail & Related papers  (2023-10-26T14:37:01Z)
• The Sem-Lex Benchmark: Modeling ASL Signs and Their Phonemes [6.0179345110920455]
  We introduce a new resource for American Sign Language (ASL) modeling, the Sem-Lex Benchmark. The Benchmark is the current largest of its kind, consisting of over 84k videos of isolated sign productions from deaf ASL signers who gave informed consent and received compensation. We present a suite of experiments which make use of the linguistic information in ASL-LEX, evaluating the practicality and fairness of the Sem-Lex Benchmark for isolated sign recognition (ISR)
  arXiv  Detail & Related papers  (2023-09-30T00:25:43Z)
• Slovo: Russian Sign Language Dataset [83.93252084624997]
  This paper presents the Russian Sign Language (RSL) video dataset Slovo, produced using crowdsourcing platforms. The dataset contains 20,000 FullHD recordings, divided into 1,000 classes of isolated RSL gestures received by 194 signers.
  arXiv  Detail & Related papers  (2023-05-23T21:00:42Z)
• LSA-T: The first continuous Argentinian Sign Language dataset for Sign Language Translation [52.87578398308052]
  Sign language translation (SLT) is an active field of study that encompasses human-computer interaction, computer vision, natural language processing and machine learning. This paper presents the first continuous Argentinian Sign Language (LSA) dataset. It contains 14,880 sentence level videos of LSA extracted from the CN Sordos YouTube channel with labels and keypoints annotations for each signer.
  arXiv  Detail & Related papers  (2022-11-14T14:46:44Z)
• ASL-Homework-RGBD Dataset: An annotated dataset of 45 fluent and non-fluent signers performing American Sign Language homeworks [32.3809065803553]
  This dataset contains videos of fluent and non-fluent signers using American Sign Language (ASL) A total of 45 fluent and non-fluent participants were asked to perform signing homework assignments. The data is annotated to identify several aspects of signing including grammatical features and non-manual markers.
  arXiv  Detail & Related papers  (2022-07-08T17:18:49Z)
• ASL Video Corpora & Sign Bank: Resources Available through the American Sign Language Linguistic Research Project (ASLLRP) [0.0]
  The American Sign Language Linguistic Research Project (ASLLRP) provides Internet access to high-quality ASL video data. The manual and non-manual components of the signing have been linguistically annotated using SignStream(R)
  arXiv  Detail & Related papers  (2022-01-19T22:48:36Z)
• Skeleton Based Sign Language Recognition Using Whole-body Keypoints [71.97020373520922]
  Sign language is used by deaf or speech impaired people to communicate. Skeleton-based recognition is becoming popular that it can be further ensembled with RGB-D based method to achieve state-of-the-art performance. Inspired by the recent development of whole-body pose estimation citejin 2020whole, we propose recognizing sign language based on the whole-body key points and features.
  arXiv  Detail & Related papers  (2021-03-16T03:38:17Z)
• BSL-1K: Scaling up co-articulated sign language recognition using mouthing cues [106.21067543021887]
  We show how to use mouthing cues from signers to obtain high-quality annotations from video data. The BSL-1K dataset is a collection of British Sign Language (BSL) signs of unprecedented scale.
  arXiv  Detail & Related papers  (2020-07-23T16:59:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.