An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages

• URL: http://arxiv.org/abs/2604.02596v2
• Date: Mon, 06 Apr 2026 04:46:05 GMT
• Title: An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages
• Abstract summary: In-context learning (ICL) allows large language models to adapt to new tasks from a few examples.<n>Recent work on many-shot ICL suggests that modern LLMs can further benefit from larger ICL examples enabled by their long context windows.<n>We present an empirical study of many-shot ICL for machine translation from English into ten truly low-resource languages.
• Score: 10.94905796367051
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In-context learning (ICL) allows large language models (LLMs) to adapt to new tasks from a few examples, making it promising for languages underrepresented in pre-training. Recent work on many-shot ICL suggests that modern LLMs can further benefit from larger ICL examples enabled by their long context windows. However, such gains depend on careful example selection, and the inference cost can be prohibitive for low-resource language communities. In this paper, we present an empirical study of many-shot ICL for machine translation from English into ten truly low-resource languages recently added to FLORES+. We analyze the effects of retrieving more informative examples, using out-of-domain data, and ordering examples by length. Our findings show that many-shot ICL becomes more effective as the number of examples increases. More importantly, we show that BM25-based retrieval substantially improves data efficiency: 50 retrieved examples roughly match 250 many-shot examples, while 250 retrieved examples perform similarly to 1,000 many-shot examples.

Related papers

• When Many-Shot Prompting Fails: An Empirical Study of LLM Code Translation [0.23332469289621782]
  Large Language Models (LLMs) with vast context windows offer new avenues for in-context learning (ICL)<n>We systematically evaluate the impact of scaling in-context examples from zero-shot to many-shot configurations of up to 625 examples.<n>Our findings reveal a "many-shot paradox": while static similarity metrics may modestly improve with more examples, functional correctness consistently peaks with few-shot prompting.
  arXiv  Detail & Related papers  (2025-10-19T12:29:13Z)
• Revisiting In-Context Learning with Long Context Language Models [26.141121450077637]
  In-Context Learning (ICL) is a technique by which language models make predictions based on examples provided in their input context.<n>The advent of Long Context Language Models (LCLMs) has significantly increased the number of examples that can be included in context.<n>This paper explores whether ICL performance in a many-shot regime is still sensitive to the method of sample selection.
  arXiv  Detail & Related papers  (2024-12-22T08:55:19Z)
• PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from Related Example Banks [57.86928556668849]
  Large Language Models (LLMs) have recently demonstrated impressive few-shot learning capabilities through in-context learning (ICL)<n>ICL performance is highly dependent on the choice of few-shot demonstrations, making the selection of the most optimal examples a persistent research challenge.<n>In this work, we propose PromptRefine, a novel Alternating Minimization approach for example selection that improves ICL performance on low-resource Indic languages.
  arXiv  Detail & Related papers  (2024-12-07T17:51:31Z)
• Generating bilingual example sentences with large language models as lexicography assistants [2.6550899846546527]
  We present a study of LLMs' performance in generating and rating example sentences for bilingual dictionaries across languages with varying resource levels. We evaluate the quality of LLM-generated examples against the GDEX (Good Dictionary EXample) criteria: typicality, informativeness, and intelligibility.
  arXiv  Detail & Related papers  (2024-10-04T06:45:48Z)
• XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples [64.79218405438871]
  We introduce XAMPLER: Cross-Lingual Example Retrieval, a method tailored to tackle the challenge of cross-lingual in-context learning.<n>XAMPLER first trains a retriever based on Glot500, a multilingual small language model.<n>It can directly retrieve English examples as few-shot examples for in-context learning of target languages.
  arXiv  Detail & Related papers  (2024-05-08T15:13:33Z)
• Many-Shot In-Context Learning [58.395589302800566]
  Large language models (LLMs) excel at few-shot in-context learning (ICL) We observe significant performance gains across a wide variety of generative and discriminative tasks. Unlike few-shot learning, many-shot learning is effective at overriding pretraining biases.
  arXiv  Detail & Related papers  (2024-04-17T02:49:26Z)
• ParaICL: Towards Parallel In-Context Learning [74.38022919598443]
  Large language models (LLMs) have become the norm in natural language processing.<n>Few-shot in-context learning (ICL) relies on the choice of few-shot demonstration examples.<n>We propose a novel method named parallel in-context learning (ParaICL)
  arXiv  Detail & Related papers  (2024-03-31T05:56:15Z)
• CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models [53.9835961434552]
  We introduce the Chinese Instruction-Following Benchmark (CIF-Bench) to evaluate the generalizability of large language models (LLMs) to the Chinese language. CIF-Bench comprises 150 tasks and 15,000 input-output pairs, developed by native speakers to test complex reasoning and Chinese cultural nuances. To mitigate data contamination, we release only half of the dataset publicly, with the remainder kept private, and introduce diversified instructions to minimize score variance.
  arXiv  Detail & Related papers  (2024-02-20T16:02:12Z)
• Learning to Retrieve In-Context Examples for Large Language Models [69.9707552694766]
  Large language models (LLMs) have demonstrated their ability to learn in-context. The effectiveness of in-context learning is heavily reliant on the quality of the selected examples. We propose a novel framework to iteratively train dense retrievers that can identify high-quality in-context examples.
  arXiv  Detail & Related papers  (2023-07-14T05:23:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.