Application of Transfer Learning to Sign Language Recognition using an Inflated 3D Deep Convolutional Neural Network

• URL: http://arxiv.org/abs/2103.05111v1
• Date: Thu, 25 Feb 2021 13:37:39 GMT
• Title: Application of Transfer Learning to Sign Language Recognition using an Inflated 3D Deep Convolutional Neural Network
• Authors: Roman T\"ongi
• Abstract summary: Transfer learning is a technique to utilize a related task with an abundance of data available to help solve a target task lacking sufficient data. This paper investigates how effectively transfer learning can be applied to isolated sign language recognition.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sign language is the primary language for people with a hearing loss. Sign language recognition (SLR) is the automatic recognition of sign language, which represents a challenging problem for computers, though some progress has been made recently using deep learning. Huge amounts of data are generally required to train deep learning models. However, corresponding datasets are missing for the majority of sign languages. Transfer learning is a technique to utilize a related task with an abundance of data available to help solve a target task lacking sufficient data. Transfer learning has been applied highly successfully in computer vision and natural language processing. However, much less research has been conducted in the field of SLR. This paper investigates how effectively transfer learning can be applied to isolated SLR using an inflated 3D convolutional neural network as the deep learning architecture. Transfer learning is implemented by pre-training a network on the American Sign Language dataset MS-ASL and subsequently fine-tuning it separately on three different sizes of the German Sign Language dataset SIGNUM. The results of the experiments give clear empirical evidence that transfer learning can be effectively applied to isolated SLR. The accuracy performances of the networks applying transfer learning increased substantially by up to 21% as compared to the baseline models that were not pre-trained on the MS-ASL dataset.

Related papers

• Deep Neural Network-Based Sign Language Recognition: A Comprehensive Approach Using Transfer Learning with Explainability [0.0]
  We suggest a novel solution that uses a deep neural network to fully automate sign language recognition. This methodology integrates sophisticated preprocessing methodologies to optimise the overall performance. Our model's ability to provide informational clarity was assessed using the SHAP (SHapley Additive exPlanations) method.
  arXiv  Detail & Related papers  (2024-09-11T17:17:44Z)
• Transfer Learning for Cross-dataset Isolated Sign Language Recognition in Under-Resourced Datasets [2.512406961007489]
  We use a temporal graph convolution-based sign language recognition approach to evaluate five supervised transfer learning approaches. Experiments demonstrate that improvement over finetuning based transfer learning is possible with specialized supervised transfer learning methods.
  arXiv  Detail & Related papers  (2024-03-21T16:36:40Z)
• Neural Sign Actors: A diffusion model for 3D sign language production from text [51.81647203840081]
  Sign Languages (SL) serve as the primary mode of communication for the Deaf and Hard of Hearing communities. This work makes an important step towards realistic neural sign avatars, bridging the communication gap between Deaf and hearing communities.
  arXiv  Detail & Related papers  (2023-12-05T12:04:34Z)
• Multimodal Masked Autoencoders Learn Transferable Representations [127.35955819874063]
  We propose a simple and scalable network architecture, the Multimodal Masked Autoencoder (M3AE) M3AE learns a unified encoder for both vision and language data via masked token prediction. We provide an empirical study of M3AE trained on a large-scale image-text dataset, and find that M3AE is able to learn generalizable representations that transfer well to downstream tasks.
  arXiv  Detail & Related papers  (2022-05-27T19:09:42Z)
• A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation [54.29679610921429]
  Existing sign language datasets contain only about 10K-20K pairs of sign videos, gloss annotations and texts. Data is thus a bottleneck for training effective sign language translation models. This simple baseline surpasses the previous state-of-the-art results on two sign language translation benchmarks.
  arXiv  Detail & Related papers  (2022-03-08T18:59:56Z)
• Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
  Sign language is commonly used by deaf or mute people to communicate. We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR) Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
  arXiv  Detail & Related papers  (2021-10-12T16:57:18Z)
• Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition [54.92161571089808]
  Cross-lingual NER transfers knowledge from rich-resource language to languages with low resources. Existing cross-lingual NER methods do not make good use of rich unlabeled data in target languages. We develop a novel approach based on the ideas of semi-supervised learning and reinforcement learning.
  arXiv  Detail & Related papers  (2021-06-01T05:46:22Z)
• Interpretation of Swedish Sign Language using Convolutional Neural Networks and Transfer Learning [2.7629216089139934]
  We use Convolutional Neural Networks (CNNs) and transfer learning in order to make computers able to interpret signs of the Swedish Sign Language (SSL) hand alphabet. Our model consist of the implementation of a pre-trained InceptionV3 network, and the usage of the mini-batch gradient descent optimization algorithm. The final accuracy of the model, based on 8 study subjects and 9,400 images, is 85%.
  arXiv  Detail & Related papers  (2020-10-15T15:34:09Z)
• Transfer Learning for British Sign Language Modelling [0.0]
  Research in minority languages, including sign languages, is hampered by the severe lack of data. This has led to work on transfer learning methods, whereby a model developed for one language is reused as the starting point for a model on a second language. In this paper, we examine two transfer learning techniques of fine-tuning and layer substitution for language modelling of British Sign Language.
  arXiv  Detail & Related papers  (2020-06-03T10:13:29Z)
• Meta-Transfer Learning for Code-Switched Speech Recognition [72.84247387728999]
  We propose a new learning method, meta-transfer learning, to transfer learn on a code-switched speech recognition system in a low-resource setting. Our model learns to recognize individual languages, and transfer them so as to better recognize mixed-language speech by conditioning the optimization on the code-switching data.
  arXiv  Detail & Related papers  (2020-04-29T14:27:19Z)
• ASL Recognition with Metric-Learning based Lightweight Network [0.0]
  We propose a lightweight network for ASL gesture recognition with a performance sufficient for practical applications. The training code is available as part of Intel OpenVINO Training Extensions.
  arXiv  Detail & Related papers  (2020-04-10T14:41:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.