ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directions

• URL: http://arxiv.org/abs/2401.06804v1
• Date: Wed, 10 Jan 2024 13:39:49 GMT
• Title: ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directions
• Authors: Nada Shahin and Leila Ismail
• Abstract summary: ChatGPT is a language model based on Generative AI. It can translate from English to American (ASL), Australian (AUSLAN), and British (BSL) sign languages and from Arabic Sign Language (ArSL) to English with only one prompt iteration. However, the model failed to translate from Arabic to ArSL and ASL, AUSLAN, and BSL to Arabic.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: ChatGPT is a language model based on Generative AI. Existing research work on ChatGPT focused on its use in various domains. However, its potential for Sign Language Translation (SLT) is yet to be explored. This paper addresses this void. Therefore, we present GPT's evolution aiming a retrospective analysis of the improvements to its architecture for SLT. We explore ChatGPT's capabilities in translating different sign languages in paving the way to better accessibility for deaf and hard-of-hearing community. Our experimental results indicate that ChatGPT can accurately translate from English to American (ASL), Australian (AUSLAN), and British (BSL) sign languages and from Arabic Sign Language (ArSL) to English with only one prompt iteration. However, the model failed to translate from Arabic to ArSL and ASL, AUSLAN, and BSL to Arabic. Consequently, we present challenges and derive insights for future research directions.

Related papers

• Scaling up Multimodal Pre-training for Sign Language Understanding [96.17753464544604]
  Sign language serves as the primary meaning of communication for the deaf-mute community. To facilitate communication between the deaf-mute and hearing people, a series of sign language understanding (SLU) tasks have been studied. These tasks investigate sign language topics from diverse perspectives and raise challenges in learning effective representation of sign language videos.
  arXiv  Detail & Related papers  (2024-08-16T06:04:25Z)
• Primacy Effect of ChatGPT [69.49920102917598]
  We study the primacy effect of ChatGPT: the tendency of selecting the labels at earlier positions as the answer. We hope that our experiments and analyses provide additional insights into building more reliable ChatGPT-based solutions.
  arXiv  Detail & Related papers  (2023-10-20T00:37:28Z)
• GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP [21.6253870440136]
  This study conducts a large-scale automated and human evaluation of ChatGPT, encompassing 44 distinct language understanding and generation tasks. Our findings indicate that, despite its remarkable performance in English, ChatGPT is consistently surpassed by smaller models that have undergone finetuning on Arabic.
  arXiv  Detail & Related papers  (2023-05-24T10:12:39Z)
• Phoenix: Democratizing ChatGPT across Languages [68.75163236421352]
  We release a large language model "Phoenix", achieving competitive performance among open-source English and Chinese models. We believe this work will be beneficial to make ChatGPT more accessible, especially in countries where people cannot use ChatGPT due to restrictions from OpenAI or local goverments.
  arXiv  Detail & Related papers  (2023-04-20T16:50:04Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine [97.8609714773255]
  We evaluate ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. ChatGPT performs competitively with commercial translation products but lags behind significantly on low-resource or distant languages. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly boosted.
  arXiv  Detail & Related papers  (2023-01-20T08:51:36Z)
• All You Need In Sign Language Production [50.3955314892191]
  Sign language recognition and production need to cope with some critical challenges. We present an introduction to the Deaf culture, Deaf centers, psychological perspective of sign language. Also, the backbone architectures and methods in SLP are briefly introduced and the proposed taxonomy on SLP is presented.
  arXiv  Detail & Related papers  (2022-01-05T13:45:09Z)
• Modeling Global Body Configurations in American Sign Language [2.8575516056239576]
  American Sign Language (ASL) is the fourth most commonly used language in the United States. ASL is the language most commonly used by Deaf people in the United States and the English-speaking regions of Canada.
  arXiv  Detail & Related papers  (2020-09-03T06:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.