Lius: Translation Model Based Instructional Lingustic Using Continual Instruction Tuning In Kupang Malay

• URL: http://arxiv.org/abs/2606.11786v1
• Date: Wed, 10 Jun 2026 08:20:09 GMT
• Title: Lius: Translation Model Based Instructional Lingustic Using Continual Instruction Tuning In Kupang Malay
• Abstract summary: Large Language Models (LLMs) offer new potential for translation tasks but often experience performance degradation when handling low-resource languages.<n>We propose an approach for fine-tuning LLMs on a low-resource language, Kupang Malay.
• Score: 0.09384603486206743
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large Language Models (LLMs) offer new potential for translation tasks but often experience performance degradation when handling low-resource languages. To address this limitation, we propose an approach for fine-tuning LLMs on a low-resource language, Kupang Malay. Our approach involves designing a set of instructions by leveraging explicit lexical and semantic features from a bilingual dictionary, and introducing Continual Instruction Tuning (CIT), a training paradigm that enables iterative instruction-based training. Experimental results demonstrate that our model, named Lius, yields notable improvements over standard instruction-tuned models by outperforming 4-6 points, and surpassing both Neural Machine Translation (NMT) and Multilingual LLM models by 10-13 points on several evaluation metrics. These findings highlight the potential of our approach to mitigate the reliance on large-scale parallel data in low-resource language translation.

Related papers

• Improving Low-Resource Translation with Dictionary-Guided Fine-Tuning and RL: A Spanish-to-Wayuunaiki Study [0.27998963147546146]
  We propose a novel approach that enhances translation for low-resource languages by integrating an external dictionary tool and training models end-to-end.<n>We frame translation as a tool-augmented decision-making problem in which the model can selectively consult a bilingual dictionary during generation.<n>Preliminary results show that our tool-augmented models achieve up to +3.37 BLEU improvement over previous work.
  arXiv  Detail & Related papers  (2025-08-26T23:50:55Z)
• Challenges in Adapting Multilingual LLMs to Low-Resource Languages using LoRA PEFT Tuning [0.4194295877935868]
  This study investigates the effects of Low-Rank Adaptation (LoRA) -Efficient Fine-Tuning (PEFT) on multilingual Gemma models for Marathi.<n>Using a translated dataset with 52,000 instruction-response pairs, our findings reveal that while evaluation performance decline post-fine-tuning, manual assessments frequently suggest that the fine-tuned models outperform their original counterparts.
  arXiv  Detail & Related papers  (2024-11-27T18:14:38Z)
• Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptation [70.22782550540714]
  Quantization-aware Scale LeArning method based on multimodal Warmup, termed QSLAW. We introduce a Quantization-aware Scale LeArning method based on multimodal Warmup, termed QSLAW.
  arXiv  Detail & Related papers  (2024-08-07T12:42:09Z)
• TasTe: Teaching Large Language Models to Translate through Self-Reflection [82.83958470745381]
  Large language models (LLMs) have exhibited remarkable performance in various natural language processing tasks. We propose the TasTe framework, which stands for translating through self-reflection. The evaluation results in four language directions on the WMT22 benchmark reveal the effectiveness of our approach compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-12T17:21:21Z)
• Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuning [57.323716555996114]
  Off-target translation remains an unsolved problem, especially for low-resource languages. Recent works have either designed advanced prompting strategies to highlight the functionality of translation instructions or exploited the in-context learning ability of LLMs. In this work, we design a two-stage fine-tuning algorithm to improve the instruction-following ability (especially the translation direction) of LLMs.
  arXiv  Detail & Related papers  (2024-03-21T13:47:40Z)
• TIM: Teaching Large Language Models to Translate with Comparison [78.66926087162672]
  We propose a novel framework using examples in comparison to teach LLMs to learn translation. Our approach involves presenting the model with examples of correct and incorrect translations and using a preference loss to guide the model's learning. Our findings offer a new perspective on fine-tuning LLMs for translation tasks and provide a promising solution for generating high-quality translations.
  arXiv  Detail & Related papers  (2023-07-10T08:15:40Z)
• Exploiting Multilingualism in Low-resource Neural Machine Translation via Adversarial Learning [3.2258463207097017]
  Generative Adversarial Networks (GAN) offer a promising approach for Neural Machine Translation (NMT) In GAN, similar to bilingual models, multilingual NMT only considers one reference translation for each sentence during model training. This article proposes Denoising Adversarial Auto-encoder-based Sentence Interpolation (DAASI) approach to perform sentence computation.
  arXiv  Detail & Related papers  (2023-03-31T12:34:14Z)
• Multilingual Neural Machine Translation:Can Linguistic Hierarchies Help? [29.01386302441015]
  Multilingual Neural Machine Translation (MNMT) trains a single NMT model that supports translation between multiple languages. The performance of an MNMT model is highly dependent on the type of languages used in training, as transferring knowledge from a diverse set of languages degrades the translation performance due to negative transfer. We propose a Hierarchical Knowledge Distillation (HKD) approach for MNMT which capitalises on language groups generated according to typological features and phylogeny of languages to overcome the issue of negative transfer.
  arXiv  Detail & Related papers  (2021-10-15T02:31:48Z)
• Improving Multilingual Translation by Representation and Gradient Regularization [82.42760103045083]
  We propose a joint approach to regularize NMT models at both representation-level and gradient-level. Our results demonstrate that our approach is highly effective in both reducing off-target translation occurrences and improving zero-shot translation performance.
  arXiv  Detail & Related papers  (2021-09-10T10:52:21Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.