ERUPD -- English to Roman Urdu Parallel Dataset

• URL: http://arxiv.org/abs/2412.17562v1
• Date: Mon, 23 Dec 2024 13:33:09 GMT
• Title: ERUPD -- English to Roman Urdu Parallel Dataset
• Authors: Mohammed Furqan, Raahid Bin Khaja, Rayyan Habeeb,
• Abstract summary: Roman Urdu is a Latin-script adaptation of Urdu widely used in digital communication. This study creates a novel parallel dataset comprising 75,146 sentence pairs.
• Score: 0.0
• License:
• Abstract: Bridging linguistic gaps fosters global growth and cultural exchange. This study addresses the challenges of Roman Urdu -- a Latin-script adaptation of Urdu widely used in digital communication -- by creating a novel parallel dataset comprising 75,146 sentence pairs. Roman Urdu's lack of standardization, phonetic variability, and code-switching with English complicates language processing. We tackled this by employing a hybrid approach that combines synthetic data generated via advanced prompt engineering with real-world conversational data from personal messaging groups. We further refined the dataset through a human evaluation phase, addressing linguistic inconsistencies and ensuring accuracy in code-switching, phonetic representations, and synonym variability. The resulting dataset captures Roman Urdu's diverse linguistic features and serves as a critical resource for machine translation, sentiment analysis, and multilingual education.

Related papers

• RetrieveGPT: Merging Prompts and Mathematical Models for Enhanced Code-Mixed Information Retrieval [0.0]
  In India, social media users frequently engage in code-mixed conversations using the Roman script. This paper focuses on the challenges of extracting relevant information from code-mixed conversations. We develop a mechanism to automatically identify the most relevant answers from code-mixed conversations.
  arXiv  Detail & Related papers  (2024-11-07T14:41:01Z)
• CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving [61.73180469072787]
  We focus on the problem of spoken translation (ST) of code-switched speech in Indian languages to English text. We present a new end-to-end model architecture COSTA that scaffolds on pretrained automatic speech recognition (ASR) and machine translation (MT) modules. COSTA significantly outperforms many competitive cascaded and end-to-end multimodal baselines by up to 3.5 BLEU points.
  arXiv  Detail & Related papers  (2024-06-16T16:10:51Z)
• Open the Data! Chuvash Datasets [50.59120569845975]
  We introduce four comprehensive datasets for the Chuvash language. These datasets include a monolingual dataset, a parallel dataset with Russian, a parallel dataset with English, and an audio dataset.
  arXiv  Detail & Related papers  (2024-05-31T07:51:19Z)
• CORI: CJKV Benchmark with Romanization Integration -- A step towards Cross-lingual Transfer Beyond Textual Scripts [50.44270798959864]
  Some languages are more well-connected than others, and target languages can benefit from transferring from closely related languages. We study the impact of source language for cross-lingual transfer, demonstrating the importance of selecting source languages that have high contact with the target language.
  arXiv  Detail & Related papers  (2024-04-19T04:02:50Z)
• Language Detection for Transliterated Content [0.0]
  We study the widespread use of transliteration, where the English alphabet is employed to convey messages in native languages. This paper addresses this challenge through a dataset of phone text messages in Hindi and Russian transliterated into English. The research pioneers innovative approaches to identify and convert transliterated text.
  arXiv  Detail & Related papers  (2024-01-09T15:40:54Z)
• Multi-lingual and Multi-cultural Figurative Language Understanding [69.47641938200817]
  Figurative language permeates human communication, but is relatively understudied in NLP. We create a dataset for seven diverse languages associated with a variety of cultures: Hindi, Indonesian, Javanese, Kannada, Sundanese, Swahili and Yoruba. Our dataset reveals that each language relies on cultural and regional concepts for figurative expressions, with the highest overlap between languages originating from the same region. All languages exhibit a significant deficiency compared to English, with variations in performance reflecting the availability of pre-training and fine-tuning data.
  arXiv  Detail & Related papers  (2023-05-25T15:30:31Z)
• $\mu$PLAN: Summarizing using a Content Plan as Cross-Lingual Bridge [72.64847925450368]
  Cross-lingual summarization consists of generating a summary in one language given an input document in a different language. This work presents $mu$PLAN, an approach to cross-lingual summarization that uses an intermediate planning step as a cross-lingual bridge.
  arXiv  Detail & Related papers  (2023-05-23T16:25:21Z)
• Romanization-based Large-scale Adaptation of Multilingual Language Models [124.57923286144515]
  Large multilingual pretrained language models (mPLMs) have become the de facto state of the art for cross-lingual transfer in NLP. We study and compare a plethora of data- and parameter-efficient strategies for adapting the mPLMs to romanized and non-romanized corpora of 14 diverse low-resource languages. Our results reveal that UROMAN-based transliteration can offer strong performance for many languages, with particular gains achieved in the most challenging setups.
  arXiv  Detail & Related papers  (2023-04-18T09:58:34Z)
• HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text [1.6675267471157407]
  We present a corpus (HinGE) for a widely popular code-mixed language Hinglish (code-mixing of Hindi and English languages) HinGE has Hinglish sentences generated by humans as well as two rule-based algorithms corresponding to the parallel Hindi-English sentences. In addition, we demonstrate the inefficacy of widely-used evaluation metrics on the code-mixed data.
  arXiv  Detail & Related papers  (2021-07-08T11:11:37Z)
• NUIG-Shubhanker@Dravidian-CodeMix-FIRE2020: Sentiment Analysis of Code-Mixed Dravidian text using XLNet [0.0]
  Social media has penetrated into multilingual societies, however most of them use English to be a preferred language for communication. It looks natural for them to mix their cultural language with English during conversations resulting in abundance of multilingual data, call this code-mixed data, available in todays' world. Downstream NLP tasks using such data is challenging due to the semantic nature of it being spread across multiple languages. This paper uses an auto-regressive XLNet model to perform sentiment analysis on code-mixed Tamil-English and Malayalam-English datasets.
  arXiv  Detail & Related papers  (2020-10-15T14:09:02Z)
• A Clustering Framework for Lexical Normalization of Roman Urdu [10.746384310607157]
  Roman Urdu is an informal form of the Urdu language written in Roman script. It lacks standard spelling and hence poses several normalization challenges during automatic language processing. We present a feature-based clustering framework for the lexical normalization of Roman Urdu corpora.
  arXiv  Detail & Related papers  (2020-03-31T20:21:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.