Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition

• URL: http://arxiv.org/abs/2606.19170v1
• Date: Wed, 17 Jun 2026 15:13:19 GMT
• Title: Dango: A Strictly L1-Only Large Language Model for Studying Second Language Acquisition
• Abstract summary: Dango is a large language model designed for controlled studies of L1-to-L2 transfer in second language acquisition (SLA)<n>We identify a key challenge when scaling models to this size: L2 contamination within the "monolingual" pretraining corpus used for L1 acquisition.<n>To address this, we propose a filtering method to reduce premature exposure to English while preserving realistic, minimal exposure.
• Score: 13.572594864092096
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We introduce Dango, a 1.8B-parameter large language model designed for controlled studies of L1-to-L2 (Japanese-to-English) transfer in second language acquisition (SLA). While previous studies have explored SLA in language models, they have predominantly relied on smaller or non-decoder models, limiting their ability to generate open-ended text and reducing their suitability as practical L2 simulators. We identify a key challenge when scaling models to this size: L2 contamination within the "monolingual" pretraining corpus used for L1 acquisition. To address this, we propose a filtering method to reduce premature exposure to English while preserving realistic, minimal exposure. We then fine-tune the model on LLM-generated L2-learning lessons to simulate the L2 acquisition process. Our evaluations confirm that Dango develops human-like L2 production patterns, outperforming both unfiltered and standard multilingual baselines. We release the model, data, and code to facilitate reproducible computational SLA research and learner-facing applications.

Related papers

• Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases [22.048949559200935]
  This study evaluates Large Language Models' ability to simulate non-native-like English use observed in human second language (L2) learners.<n>In dialogue-based interviews, we prompt LLMs to mimic L2 English learners with specific L1s across seven languages.<n>Our analysis examines L1-driven linguistic biases, such as reference word usage and avoidance behaviors, using information-theoretic and distributional density measures.
  arXiv  Detail & Related papers  (2025-02-20T12:34:46Z)
• Enhancing Code Generation for Low-Resource Languages: No Silver Bullet [55.39571645315926]
  Large Language Models (LLMs) rely on large and diverse datasets to learn syntax, semantics, and usage patterns of programming languages.<n>For low-resource languages, the limited availability of such data hampers the models' ability to generalize effectively.<n>We present an empirical study investigating the effectiveness of several approaches for boosting LLMs' performance on low-resource languages.
  arXiv  Detail & Related papers  (2025-01-31T12:23:28Z)
• Language Portability Strategies for Open-domain Dialogue with Pre-trained Language Models from High to Low Resource Languages [1.7436854281619139]
  We propose a study of linguistic portability strategies of large pre-trained language models (PLMs) used for open-domain dialogue systems. In particular the target low-resource language (L_T) will be simulated with French, as it lacks of task-specific resources.
  arXiv  Detail & Related papers  (2024-07-01T14:20:54Z)
• Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean [3.4735169184479524]
  Large language models (LLMs) use pretraining to predict the subsequent word; however, their expansion requires significant computing resources. This study proposed three strategies to enhance the performance of LRLs based on the publicly available MLLMs.
  arXiv  Detail & Related papers  (2024-03-16T10:26:38Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• YAYI 2: Multilingual Open-Source Large Language Models [53.92832054643197]
  We propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback.
  arXiv  Detail & Related papers  (2023-12-22T17:34:47Z)
• The Ups and Downs of Large Language Model Inference with Vocabulary Trimming by Language Heuristics [74.99898531299148]
  This research examines vocabulary trimming (VT) inspired by restricting embedding entries to the language of interest to bolster time and memory efficiency. We apply two languages to trim the full vocabulary - Unicode-based script filtering and corpus-based selection - to different language families and sizes. It is found that VT reduces the memory usage of small models by nearly 50% and has an upper bound of 25% improvement in generation speed.
  arXiv  Detail & Related papers  (2023-11-16T09:35:50Z)
• L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language Models [102.00201523306986]
  We present L2CEval, a systematic evaluation of the language-to-code generation capabilities of large language models (LLMs) We analyze the factors that potentially affect their performance, such as model size, pretraining data, instruction tuning, and different prompting methods. In addition to assessing model performance, we measure confidence calibration for the models and conduct human evaluations of the output programs.
  arXiv  Detail & Related papers  (2023-09-29T17:57:00Z)
• SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT [0.0]
  Cross-linguistic transfer is the influence of linguistic structure of a speaker's native language on the successful acquisition of a foreign language. We find that NLP literature has not given enough attention to the phenomenon of negative transfer. Our findings call for further research using our novel Transformer-based SLA models.
  arXiv  Detail & Related papers  (2023-05-31T06:22:07Z)
• A Survey of Large Language Models [81.06947636926638]
  Language modeling has been widely studied for language understanding and generation in the past two decades. Recently, pre-trained language models (PLMs) have been proposed by pre-training Transformer models over large-scale corpora. To discriminate the difference in parameter scale, the research community has coined the term large language models (LLM) for the PLMs of significant size.
  arXiv  Detail & Related papers  (2023-03-31T17:28:46Z)
• Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding [6.68194398006805]
  We investigate fine-tuning of a pre-trained wav2vec 2.0 model citebaevski2020wav2vec,xu2021self under a rich set of L1 and L2 training conditions. We find that while the large self-trained wav2vec 2.0 may be internalizing sufficient decoding knowledge for clean L1 speech, this does not hold for L2 speech.
  arXiv  Detail & Related papers  (2022-02-10T18:13:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.