Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment

• URL: http://arxiv.org/abs/2508.04865v1
• Date: Wed, 06 Aug 2025 20:30:55 GMT
• Title: Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environment
• Authors: Aleksander Boruch-Gruszecki, Yangtian Zi, Zixuan Wu, Tejas Oberoi, Carolyn Jane Anderson, Joydeep Biswas, Arjun Guha,
• Abstract summary: Agnostics is a language-agnostic post-training pipeline that eliminates per-language engineering.<n>Key idea is to judge code solely by its externally observable behavior, so a single verifier can test solutions written in any language.<n>We will release the language-agnostic training datasets (Ag-MBPP-X, Ag-Codeforces-X, Ag-LiveCodeBench-X), training code, and ready-to-use configurations, making RL post-training in any programming language as simple as editing a short YAML file.
• Score: 44.47221244109926
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) already excel at writing code in high-resource languages such as Python and JavaScript, yet stumble on low-resource languages that remain essential to science and engineering. Besides the obvious shortage of pre-training data, post-training itself is a bottleneck: every new language seems to require new datasets, test harnesses, and reinforcement-learning (RL) infrastructure. We introduce Agnostics, a language-agnostic post-training pipeline that eliminates this per-language engineering. The key idea is to judge code solely by its externally observable behavior, so a single verifier can test solutions written in any language. Concretely, we (i) use an LLM to rewrite existing unit-test datasets into an I/O format, (ii) supply a short configuration that tells the verifier how to compile and run a target language, and (iii) apply reinforcement learning with verifiable rewards (RLVR) in a robust code execution environment. Applied to five low-resource languages--Lua, Julia, R, OCaml, and Fortran--Agnostics (1) improves Qwen-3 4B to performance that rivals other 16B-70B open-weight models; (2) scales cleanly to larger and diverse model families (Qwen-3 8B, DeepSeek Coder 6.7B Instruct, Phi 4 Mini); and (3) for ${\le} 16$B parameter models, sets new state-of-the-art pass@1 results on MultiPL-E and a new multi-language version LiveCodeBench that we introduce. We will release the language-agnostic training datasets (Ag-MBPP-X, Ag-Codeforces-X, Ag-LiveCodeBench-X), training code, and ready-to-use configurations, making RL post-training in any programming language as simple as editing a short YAML file.

Related papers

• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• TinyHelen's First Curriculum: Training and Evaluating Tiny Language Models in a Simpler Language Environment [30.93798042712827]
  Training language models (LMs) and their application agents is increasingly costly due to large datasets and models.<n>We propose a pipeline to refine text data by eliminating noise, minimizing vocabulary, and maintaining genre-specific patterns.<n>Our experiments show that leaner pre-training boosts LM learning efficiency.
  arXiv  Detail & Related papers  (2024-12-31T16:08:15Z)
• LangSAMP: Language-Script Aware Multilingual Pretraining [48.16511046793275]
  We propose Language-Script Aware Multilingual Pretraining (LangSAMP)<n>LangSAMP incorporates both language and script embeddings to enhance representation learning.<n>We apply LangSAMP to the continual pretraining of XLM-R on a highly multilingual corpus covering more than 500 languages.
  arXiv  Detail & Related papers  (2024-09-26T18:29:10Z)
• CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution [50.1875460416205]
  The CRUXEVAL-X code reasoning benchmark contains 19 programming languages.<n>It comprises at least 600 subjects for each language, along with 19K content-consistent tests in total.<n>Even a model trained solely on Python can achieve at most 34.4% Pass@1 in other languages.
  arXiv  Detail & Related papers  (2024-08-23T11:43:00Z)
• An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models [31.231720803637085]
  Language Models (LMs) excel in natural language processing tasks for English but show reduced performance in most other languages. limited vocabulary coverage in the original model's tokenizer leads to inadequate representation of new languages. Constrained Word2Vec (CW2V) does not require cross-lingual embeddings.
  arXiv  Detail & Related papers  (2024-07-08T11:38:49Z)
• DocCGen: Document-based Controlled Code Generation [33.19206322891497]
  DocCGen is a framework that can leverage rich knowledge by breaking the NL-to-Code generation task for structured code languages into a two-step process. Our experiments show that DocCGen consistently improves different-sized language models across all six evaluation metrics.
  arXiv  Detail & Related papers  (2024-06-17T08:34:57Z)
• CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models [59.91221728187576]
  This paper introduces the CMU Linguistic Linguistic Backend, an open-source framework that simplifies model deployment and continuous human-in-the-loop fine-tuning of NLP models. CMULAB enables users to leverage the power of multilingual models to quickly adapt and extend existing tools for speech recognition, OCR, translation, and syntactic analysis to new languages.
  arXiv  Detail & Related papers  (2024-04-03T02:21:46Z)
• Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs [2.9242435458494445]
  This paper presents an effective approach for boosting the performance of Code LLMs on low-resource languages using semi-synthetic data. We apply this approach to generate tens of thousands of validated training items for Julia, Lua, OCaml, R, and Racket.
  arXiv  Detail & Related papers  (2023-08-19T03:19:01Z)
• COMEX: A Tool for Generating Customized Source Code Representations [7.151800146054561]
  COMEX is a framework that allows researchers and developers to create and combine multiple code-views. It can analyze both method-level snippets and program-level snippets by using both intra-procedural and inter-procedural snippets. It is built on tree-sitter - a widely used incremental analysis tool that supports over 40 languages.
  arXiv  Detail & Related papers  (2023-07-10T16:46:34Z)
• Revisiting Language Encoding in Learning Multilingual Representations [70.01772581545103]
  We propose a new approach called Cross-lingual Language Projection (XLP) to replace language embedding. XLP projects the word embeddings into language-specific semantic space, and then the projected embeddings will be fed into the Transformer model. Experiments show that XLP can freely and significantly boost the model performance on extensive multilingual benchmark datasets.
  arXiv  Detail & Related papers  (2021-02-16T18:47:10Z)
• X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models [103.75890012041366]
  Language models (LMs) have proven surprisingly successful at capturing factual knowledge. However, studies on LMs' factual representation ability have almost invariably been performed on English. We create a benchmark of cloze-style probes for 23 typologically diverse languages.
  arXiv  Detail & Related papers  (2020-10-13T05:29:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.