A comparison of pipelines for the translation of a low resource language based on transformers

• URL: http://arxiv.org/abs/2509.12514v1
• Date: Mon, 15 Sep 2025 23:36:49 GMT
• Title: A comparison of pipelines for the translation of a low resource language based on transformers
• Abstract summary: This work compares three pipelines to produce machine translators for Bambara, a Mande language spoken in Africa by about 14,188,850 people.<n>The first pipeline trains a simple transformer to translate sentences from French into Bambara.<n>The second fine-tunes LLaMA3 (3B-8B) instructor models using decoder-only architectures for French-to-Bambara translation.<n>The third pipeline uses language distillation with a student-teacher dual neural network to integrate Bambara into a pre-trained LaBSE model, which provides language-agnostic embeddings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This work compares three pipelines for training transformer-based neural networks to produce machine translators for Bambara, a Mand\`e language spoken in Africa by about 14,188,850 people. The first pipeline trains a simple transformer to translate sentences from French into Bambara. The second fine-tunes LLaMA3 (3B-8B) instructor models using decoder-only architectures for French-to-Bambara translation. Models from the first two pipelines were trained with different hyperparameter combinations to improve BLEU and chrF scores, evaluated on both test sentences and official Bambara benchmarks. The third pipeline uses language distillation with a student-teacher dual neural network to integrate Bambara into a pre-trained LaBSE model, which provides language-agnostic embeddings. A BERT extension is then applied to LaBSE to generate translations. All pipelines were tested on Dokotoro (medical) and Bayelemagaba (mixed domains). Results show that the first pipeline, although simpler, achieves the best translation accuracy (10% BLEU, 21% chrF on Bayelemagaba), consistent with low-resource translation results. On the Yiri dataset, created for this work, it achieves 33.81% BLEU and 41% chrF. Instructor-based models perform better on single datasets than on aggregated collections, suggesting they capture dataset-specific patterns more effectively.

Related papers

• Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation [0.8602553195689513]
  Large Language Models achieve strong code generation for high resource languages like Python and Java but suffer sharp performance drops on Low-Resource Programming Languages(LRPLs) such as Julia.<n>We propose a three-phase pipeline that resolves this trilemma by decoupling syntax acquisition from algorithmic reasoning.<n>Applying to Qwen3-8B, our pipeline improves pass@1 by up to +7.6 points on MultiPL-E and +14.2 points on the Agnostics LiveCodeBench for Julia compared to SOTA results.
  arXiv  Detail & Related papers  (2026-07-08T09:18:56Z)
• A Comparative Analysis of Retrieval-Augmented Generation Techniques for Bengali Standard-to-Dialect Machine Translation Using LLMs [0.3227658251731014]
  Translating from a standard language to its regional dialects is a significant NLP challenge due to scarce data and linguistic variation.<n>This paper proposes and compares two novel RAG pipelines for standard-to-dialectal Bengali translation.
  arXiv  Detail & Related papers  (2025-12-16T08:18:18Z)
• An Efficient Approach for Machine Translation on Low-resource Languages: A Case Study in Vietnamese-Chinese [1.6932009464531739]
  We proposed an approach for machine translation in low-resource languages such as Vietnamese-Chinese.<n>Our proposed method leveraged the power of the multilingual pre-trained language model (mBART) and both Vietnamese and Chinese monolingual corpus.
  arXiv  Detail & Related papers  (2025-01-31T17:11:45Z)
• Transformers for Low-Resource Languages:Is F\'eidir Linn! [2.648836772989769]
  In general, neural translation models often under perform on language pairs with insufficient training data. We demonstrate that choosing appropriate parameters leads to considerable performance improvements. A Transformer optimized model demonstrated a BLEU score improvement of 7.8 points when compared with a baseline RNN model.
  arXiv  Detail & Related papers  (2024-03-04T12:29:59Z)
• M3ST: Mix at Three Levels for Speech Translation [66.71994367650461]
  We propose Mix at three levels for Speech Translation (M3ST) method to increase the diversity of the augmented training corpus. In the first stage of fine-tuning, we mix the training corpus at three levels, including word level, sentence level and frame level, and fine-tune the entire model with mixed data. Experiments on MuST-C speech translation benchmark and analysis show that M3ST outperforms current strong baselines and achieves state-of-the-art results on eight directions with an average BLEU of 29.9.
  arXiv  Detail & Related papers  (2022-12-07T14:22:00Z)
• Revamping Multilingual Agreement Bidirectionally via Switched Back-translation for Multilingual Neural Machine Translation [107.83158521848372]
  multilingual agreement (MA) has shown its importance for multilingual neural machine translation (MNMT) We present textbfBidirectional textbfMultilingual textbfAgreement via textbfSwitched textbfBack-textbftranslation (textbfBMA-SBT) It is a novel and universal multilingual agreement framework for fine-tuning pre-trained MNMT models.
  arXiv  Detail & Related papers  (2022-09-28T09:14:58Z)
• Amortized Noisy Channel Neural Machine Translation [53.48804610779759]
  Noisy channel models have been especially effective in neural machine translation (NMT) We aim to build an amortized noisy channel NMT model such that greedily decoding from it would generate translations that maximize the same reward as translations generated using BSR.
  arXiv  Detail & Related papers  (2021-12-16T07:10:02Z)
• Improving Neural Machine Translation by Bidirectional Training [85.64797317290349]
  We present a simple and effective pretraining strategy -- bidirectional training (BiT) for neural machine translation. Specifically, we bidirectionally update the model parameters at the early stage and then tune the model normally. Experimental results show that BiT pushes the SOTA neural machine translation performance across 15 translation tasks on 8 language pairs significantly higher.
  arXiv  Detail & Related papers  (2021-09-16T07:58:33Z)
• BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation [38.017030073108735]
  We show that a tailored and suitable bilingual pre-trained language model (dubbed BiBERT) achieves state-of-the-art translation performance. Our best models achieve BLEU scores of 30.45 for En->De and 38.61 for De->En on the IWSLT'14 dataset, and 31.26 for En->De and 34.94 for De->En on the WMT'14 dataset.
  arXiv  Detail & Related papers  (2021-09-09T23:43:41Z)
• Continual Mixed-Language Pre-Training for Extremely Low-Resource Neural Machine Translation [53.22775597051498]
  We present a continual pre-training framework on mBART to effectively adapt it to unseen languages. Results show that our method can consistently improve the fine-tuning performance upon the mBART baseline. Our approach also boosts the performance on translation pairs where both languages are seen in the original mBART's pre-training.
  arXiv  Detail & Related papers  (2021-05-09T14:49:07Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.