Related papers: ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directions

ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directions

URL: http://arxiv.org/abs/2401.06804v1
Date: Wed, 10 Jan 2024 13:39:49 GMT
Title: ChatGPT, Let us Chat Sign Language: Experiments, Architectural Elements, Challenges and Research Directions
Authors: Nada Shahin and Leila Ismail
Abstract summary: ChatGPT is a language model based on Generative AI. It can translate from English to American (ASL), Australian (AUSLAN), and British (BSL) sign languages and from Arabic Sign Language (ArSL) to English with only one prompt iteration. However, the model failed to translate from Arabic to ArSL and ASL, AUSLAN, and BSL to Arabic.
Score: 0.0
License: http://creativecommons.org/licenses/by/4.0/
Abstract: ChatGPT is a language model based on Generative AI. Existing research work on ChatGPT focused on its use in various domains. However, its potential for Sign Language Translation (SLT) is yet to be explored. This paper addresses this void. Therefore, we present GPT's evolution aiming a retrospective analysis of the improvements to its architecture for SLT. We explore ChatGPT's capabilities in translating different sign languages in paving the way to better accessibility for deaf and hard-of-hearing community. Our experimental results indicate that ChatGPT can accurately translate from English to American (ASL), Australian (AUSLAN), and British (BSL) sign languages and from Arabic Sign Language (ArSL) to English with only one prompt iteration. However, the model failed to translate from Arabic to ArSL and ASL, AUSLAN, and BSL to Arabic. Consequently, we present challenges and derive insights for future research directions.

Related papers

Signs as Tokens: An Autoregressive Multilingual Sign Language Generator [55.94334001112357]
We introduce a multilingual sign language model, Signs as Tokens (SOKE), which can generate 3D sign avatars autoregressively from text inputs. To align sign language with the LM, we develop a decoupled tokenizer that discretizes continuous signs into token sequences representing various body parts. These sign tokens are integrated into the raw text vocabulary of the LM, allowing for supervised fine-tuning on sign language datasets.
arXiv Detail & Related papers (2024-11-26T18:28:09Z)
Enhanced Sign Language Translation between American Sign Language (ASL) and Indian Sign Language (ISL) Using LLMs [0.2678472239880052]
We have come up with a research that hopes to provide a bridge between the users of American Sign Language and the users of spoken language and Indian Sign Language (ISL) This framework is tasked with key challenges such as automatically dealing with gesture variability and overcoming the linguistic differences between ASL and ISL.
arXiv Detail & Related papers (2024-11-19T17:45:12Z)
Primacy Effect of ChatGPT [69.49920102917598]
We study the primacy effect of ChatGPT: the tendency of selecting the labels at earlier positions as the answer. We hope that our experiments and analyses provide additional insights into building more reliable ChatGPT-based solutions.
arXiv Detail & Related papers (2023-10-20T00:37:28Z)
GPTAraEval: A Comprehensive Evaluation of ChatGPT on Arabic NLP [21.6253870440136]
This study conducts a large-scale automated and human evaluation of ChatGPT, encompassing 44 distinct language understanding and generation tasks. Our findings indicate that, despite its remarkable performance in English, ChatGPT is consistently surpassed by smaller models that have undergone finetuning on Arabic.
arXiv Detail & Related papers (2023-05-24T10:12:39Z)
Phoenix: Democratizing ChatGPT across Languages [68.75163236421352]
We release a large language model "Phoenix", achieving competitive performance among open-source English and Chinese models. We believe this work will be beneficial to make ChatGPT more accessible, especially in countries where people cannot use ChatGPT due to restrictions from OpenAI or local goverments.
arXiv Detail & Related papers (2023-04-20T16:50:04Z)
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
arXiv Detail & Related papers (2023-04-12T05:08:52Z)
Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine [97.8609714773255]
We evaluate ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. ChatGPT performs competitively with commercial translation products but lags behind significantly on low-resource or distant languages. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly boosted.
arXiv Detail & Related papers (2023-01-20T08:51:36Z)
All You Need In Sign Language Production [50.3955314892191]
Sign language recognition and production need to cope with some critical challenges. We present an introduction to the Deaf culture, Deaf centers, psychological perspective of sign language. Also, the backbone architectures and methods in SLP are briefly introduced and the proposed taxonomy on SLP is presented.
arXiv Detail & Related papers (2022-01-05T13:45:09Z)
Modeling Global Body Configurations in American Sign Language [2.8575516056239576]
American Sign Language (ASL) is the fourth most commonly used language in the United States. ASL is the language most commonly used by Deaf people in the United States and the English-speaking regions of Canada.
arXiv Detail & Related papers (2020-09-03T06:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.