Multilingual Tourist Assistance using ChatGPT: Comparing Capabilities in Hindi, Telugu, and Kannada

• URL: http://arxiv.org/abs/2307.15376v1
• Date: Fri, 28 Jul 2023 07:52:26 GMT
• Title: Multilingual Tourist Assistance using ChatGPT: Comparing Capabilities in Hindi, Telugu, and Kannada
• Authors: Sanjana Kolar and Rohit Kumar
• Abstract summary: This research investigates the effectiveness of ChatGPT, an AI language model by OpenAI, in translating English into Hindi, Telugu, and Kannada languages. To measure the translation quality, a test set of 50 questions from diverse fields such as general knowledge, food, and travel was used. Human evaluators rated both the accuracy and fluency of translations, offering a comprehensive perspective on the language model's performance.
• Score: 1.5762281194023464
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This research investigates the effectiveness of ChatGPT, an AI language model by OpenAI, in translating English into Hindi, Telugu, and Kannada languages, aimed at assisting tourists in India's linguistically diverse environment. To measure the translation quality, a test set of 50 questions from diverse fields such as general knowledge, food, and travel was used. These were assessed by five volunteers for accuracy and fluency, and the scores were subsequently converted into a BLEU score. The BLEU score evaluates the closeness of a machine-generated translation to a human translation, with a higher score indicating better translation quality. The Hindi translations outperformed others, showcasing superior accuracy and fluency, whereas Telugu translations lagged behind. Human evaluators rated both the accuracy and fluency of translations, offering a comprehensive perspective on the language model's performance.

Related papers

• Expanding FLORES+ Benchmark for more Low-Resource Settings: Portuguese-Emakhuwa Machine Translation Evaluation [0.0]
  Emakhuwa is a low-resource language widely spoken in Mozambique. We translate dev and devtest sets from Portuguese into Emakhuwa. We detail the translation process and quality assurance measures used.
  arXiv  Detail & Related papers  (2024-08-21T09:23:20Z)
• Google Translate Error Analysis for Mental Healthcare Information: Evaluating Accuracy, Comprehensibility, and Implications for Multilingual Healthcare Communication [8.178490288773013]
  This study explores the use of Google Translate for translating mental healthcare (MHealth) information from English to Persian, Arabic, Turkish, Romanian, and Spanish. Native speakers of the target languages manually assessed the GT translations, focusing on medical terminology accuracy, comprehensibility, and critical syntactic/semantic errors. GT output analysis revealed challenges in accurately translating medical terminology, particularly in Arabic, Romanian, and Persian.
  arXiv  Detail & Related papers  (2024-02-06T14:16:32Z)
• Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation [1.25097469793837]
  ChatGPT, a cutting-edge AI-powered,can quickly generate responses on given commands. This study compared the revised Chinese to English translation texts produced by Chinese Master of Translation and Interpretation (MTI) students.
  arXiv  Detail & Related papers  (2023-09-04T14:54:39Z)
• ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning [70.57126720079971]
  Large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) This paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages.
  arXiv  Detail & Related papers  (2023-04-12T05:08:52Z)
• How to Design Translation Prompts for ChatGPT: An Empirical Study [18.678893287863033]
  ChatGPT has demonstrated surprising abilities in natural language understanding and natural language generation. We adopt several translation prompts on a wide range of translations. Our work provides empirical evidence that ChatGPT still has great potential in translations.
  arXiv  Detail & Related papers  (2023-04-05T01:17:59Z)
• Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine [97.8609714773255]
  We evaluate ChatGPT for machine translation, including translation prompt, multilingual translation, and translation robustness. ChatGPT performs competitively with commercial translation products but lags behind significantly on low-resource or distant languages. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly boosted.
  arXiv  Detail & Related papers  (2023-01-20T08:51:36Z)
• No Language Left Behind: Scaling Human-Centered Machine Translation [69.28110770760506]
  We create datasets and models aimed at narrowing the performance gap between low and high-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-11T07:33:36Z)
• Harnessing Cross-lingual Features to Improve Cognate Detection for Low-resource Languages [50.82410844837726]
  We demonstrate the use of cross-lingual word embeddings for detecting cognates among fourteen Indian languages. We evaluate our methods to detect cognates on a challenging dataset of twelve Indian languages. We observe an improvement of up to 18% points, in terms of F-score, for cognate detection.
  arXiv  Detail & Related papers  (2021-12-16T11:17:58Z)
• Cross-Lingual Training with Dense Retrieval for Document Retrieval [56.319511218754414]
  We explore different transfer techniques for document ranking from English annotations to multiple non-English languages. Experiments on the test collections in six languages (Chinese, Arabic, French, Hindi, Bengali, Spanish) from diverse language families. We find that weakly-supervised target language transfer yields competitive performances against the generation-based target language transfer.
  arXiv  Detail & Related papers  (2021-09-03T17:15:38Z)
• A Set of Recommendations for Assessing Human-Machine Parity in Language Translation [87.72302201375847]
  We reassess Hassan et al.'s investigation into Chinese to English news translation. We show that the professional human translations contained significantly fewer errors.
  arXiv  Detail & Related papers  (2020-04-03T17:49:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.