Sample-Efficient Language Model for Hinglish Conversational AI

• URL: http://arxiv.org/abs/2504.19070v1
• Date: Sun, 27 Apr 2025 01:35:22 GMT
• Title: Sample-Efficient Language Model for Hinglish Conversational AI
• Authors: Sakshi Singh, Abhinav Prakash, Aakriti Shah, Chaitanya Sachdeva, Sanjana Dumpala,
• Abstract summary: Hinglish is a code-mixed language that combines Hindi and English.<n>The proposed approach integrates synthetically generated dialogues with insights from existing Hinglish datasets to address data scarcity.<n> Experimental results demonstrate that models with fewer parameters, when appropriately fine-tuned on high-quality code-mixed data, can achieve competitive performance.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents our process for developing a sample-efficient language model for a conversational Hinglish chatbot. Hinglish, a code-mixed language that combines Hindi and English, presents a unique computational challenge due to inconsistent spelling, lack of standardization, and limited quality of conversational data. This work evaluates multiple pre-trained cross-lingual language models, including Gemma3-4B and Qwen2.5-7B, and employs fine-tuning techniques to improve performance on Hinglish conversational tasks. The proposed approach integrates synthetically generated dialogues with insights from existing Hinglish datasets to address data scarcity. Experimental results demonstrate that models with fewer parameters, when appropriately fine-tuned on high-quality code-mixed data, can achieve competitive performance for Hinglish conversation generation while maintaining computational efficiency.

Related papers

• Mixed-Distil-BERT: Code-mixed Language Modeling for Bangla, English, and Hindi [0.0]
  We introduce Tri-Distil-BERT, a multilingual model pre-trained on Bangla, English, and Hindi, and Mixed-Distil-BERT, a model fine-tuned on code-mixed data. Our two-tiered pre-training approach offers efficient alternatives for multilingual and code-mixed language understanding.
  arXiv  Detail & Related papers  (2023-09-19T02:59:41Z)
• Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning [98.60739735409243]
  Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks. We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
  arXiv  Detail & Related papers  (2023-04-03T18:46:01Z)
• PLACES: Prompting Language Models for Social Conversation Synthesis [103.94325597273316]
  We use a small set of expert-written conversations as in-context examples to synthesize a social conversation dataset using prompting. We perform several thorough evaluations of our synthetic conversations compared to human-collected conversations.
  arXiv  Detail & Related papers  (2023-02-07T05:48:16Z)
• Investigating Effect of Dialogue History in Multilingual Task Oriented Dialogue Systems [2.695466667982714]
  Up to Dec 2021, Alexa, one of the most popular smart speakers around the world, is able to support 9 different languages. Training a virtual assistant in other languages is often more difficult, especially for those low-resource languages. We devise an efficient and effective training solution for multilingual task-orientated dialogue systems.
  arXiv  Detail & Related papers  (2021-12-23T02:27:10Z)
• Training Conversational Agents with Generative Conversational Networks [74.9941330874663]
  We use Generative Conversational Networks to automatically generate data and train social conversational agents. We evaluate our approach on TopicalChat with automatic metrics and human evaluators, showing that with 10% of seed data it performs close to the baseline that uses 100% of the data.
  arXiv  Detail & Related papers  (2021-10-15T21:46:39Z)
• Sm{\aa}prat: DialoGPT for Natural Language Generation of Swedish Dialogue by Transfer Learning [1.6111818380407035]
  State-of-the-art models for the generation of natural language dialogue have demonstrated impressive performance in simulating human-like, single-turn conversations in English. This work investigates, by an empirical study, the potential for transfer learning of such models to Swedish language.
  arXiv  Detail & Related papers  (2021-10-12T18:46:43Z)
• Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixing [19.19256927651015]
  We describe models that convert monolingual English text into Hinglish (code-mixed Hindi and English) Given the recent success of pretrained language models, we also test the utility of two recent Transformer-based encoder-decoder models. Our models place first in the overall ranking of the English-Hinglish official shared task.
  arXiv  Detail & Related papers  (2021-05-18T19:50:25Z)
• Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
  Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
  arXiv  Detail & Related papers  (2020-10-20T17:31:07Z)
• Mixed-Lingual Pre-training for Cross-lingual Summarization [54.4823498438831]
  Cross-lingual Summarization aims at producing a summary in the target language for an article in the source language. We propose a solution based on mixed-lingual pre-training that leverages both cross-lingual tasks like translation and monolingual tasks like masked language models. Our model achieves an improvement of 2.82 (English to Chinese) and 1.15 (Chinese to English) ROUGE-1 scores over state-of-the-art results.
  arXiv  Detail & Related papers  (2020-10-18T00:21:53Z)
• Plug-and-Play Conversational Models [62.77150879036442]
  We introduce an approach that does not require further computation at decoding time, while also does not require any fine-tuning of a large language model. We demonstrate, through extensive automatic and human evaluation, a high degree of control over the generated conversational responses with regard to multiple desired attributes.
  arXiv  Detail & Related papers  (2020-10-09T03:17:51Z)
• XPersona: Evaluating Multilingual Personalized Chatbot [76.00426517401894]
  We propose a multi-lingual extension of Persona-Chat, namely XPersona. Our dataset includes persona conversations in six different languages other than English for building and evaluating multilingual personalized agents.
  arXiv  Detail & Related papers  (2020-03-17T07:52:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.