Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

• URL: http://arxiv.org/abs/2605.00754v3
• Date: Fri, 08 May 2026 13:29:37 GMT
• Title: Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring
• Abstract summary: We develop Themis-CodePreference, the largest open-source collection of code preferences to date, and use it to train Themis-RM, a suite of multilingual code reward models.<n>Our experiments and ablations demonstrate positive scaling trends, strong cross-lingual transfer when training on diverse preferences.
• Score: 51.497900015187675
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Reward models (RMs) have become an indispensable fixture of the language model (LM) post-training playbook, enabling policy alignment and test-time scaling. Research on the application of RMs in code generation, however, has been comparatively sparse, with existing work largely focusing on execution feedback. This choice constrains post-training to optimizing functional correctness over self-contained executable code. In this work, we examine the training and evaluation of multilingual, multi-criteria code RMs. To this end, we first compile Themis-CodeRewardBench, a benchmark to evaluate code RMs across five preference dimensions (i.e., criteria) and eight programming languages, on which we profile 50+ code, math, and general-purpose RMs. Observing the limited proficiency of current RMs beyond scoring for functional correctness, we develop Themis-CodePreference, the largest open-source collection of code preferences to date (more than 350k preference pairs), and use it to train Themis-RM, a suite of multilingual code reward models for flexible multi-criteria scoring, ranging in size from 600M to 32B parameters. Our experiments and ablations demonstrate positive scaling trends, strong cross-lingual transfer when training on diverse preferences, and the importance of multi-criteria training for reliable code reward modeling.

Related papers

• CodeChat-Eval: Evaluating Large Language Models in Multi-Turn Code Refinement Dialogues [6.475060769782328]
  Large Language Models (LLMs) are increasingly used in software engineering to generate and refine code.<n>Existing benchmarks generally omit this multi-turn code refinement dialogue setting.<n>We introduce CodeChat-Eval, an evaluation framework that constructs evaluation sessions from multi-turn code refinement dialogues.
  arXiv  Detail & Related papers  (2026-06-24T12:16:04Z)
• Improving Code Generation via Small Language Model-as-a-judge [14.067404766521607]
  We train several state-of-the-art SLMs as code correctness judges and assess their ability to discriminate between correct and wrong implementations.<n>We show that modern SLMs outperform RankEF, even without exploiting execution-based information.
  arXiv  Detail & Related papers  (2026-02-12T13:07:36Z)
• Code-enabled language models can outperform reasoning models on diverse tasks [86.29363856881399]
  We show that standard instruct LMs can already be elicited to be strong reasoners without finetuning.<n>This is achieved by CodeAdapt, where LMs interleave natural language reasoning with code execution in a multi-step fashion.<n>We find that CodeAdapt enables three LMs to outperform the corresponding RMs on average over eight tasks.
  arXiv  Detail & Related papers  (2025-10-23T18:04:03Z)
• IFEvalCode: Controlled Code Generation [69.28317223249358]
  The paper introduces forward and backward constraints generation to improve the instruction-following capabilities of Code LLMs.<n>The authors present IFEvalCode, a multilingual benchmark comprising 1.6K test samples across seven programming languages.
  arXiv  Detail & Related papers  (2025-07-30T08:08:48Z)
• Is Compression Really Linear with Code Intelligence? [60.123628177110206]
  textitFormat Annealing is a lightweight, transparent training methodology designed to assess the intrinsic capabilities of pre-trained models equitably.<n>Our empirical results reveal a fundamental logarithmic relationship between measured code intelligence and bits-per-character (BPC)<n>Our work provides a more nuanced understanding of compression's role in developing code intelligence and contributes a robust evaluation framework in the code domain.
  arXiv  Detail & Related papers  (2025-05-16T16:59:14Z)
• Demystifying Multilingual Chain-of-Thought in Process Reward Modeling [86.98098988779809]
  We tackle the challenge of extending process reward models (PRMs) to multilingual settings.<n>We train multilingual PRMs on a dataset spanning seven languages, which is translated from English.<n>Our results highlight the sensitivity of multilingual PRMs to both the number of training languages and the volume of English data.
  arXiv  Detail & Related papers  (2025-02-18T09:11:44Z)
• The Struggles of LLMs in Cross-lingual Code Clone Detection [3.5202378300682162]
  Cross-lingual code clone detection has gained traction within the software engineering community.<n>Inspired by the significant advances in machine learning, this paper revisits cross-lingual code clone detection.<n>We evaluate the performance of five (05) Large Language Models (LLMs) and eight prompts (08) for the identification of cross-lingual code clones.
  arXiv  Detail & Related papers  (2024-08-08T12:57:14Z)
• MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration [98.18244218156492]
  Large Language Models (LLMs) have significantly advanced natural language processing.<n>As their applications expand into multi-agent environments, there arises a need for a comprehensive evaluation framework.<n>This work introduces a novel competition-based benchmark framework to assess LLMs within multi-agent settings.
  arXiv  Detail & Related papers  (2023-11-14T21:46:27Z)
• CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model [58.127534002232096]
  This paper introduces CodeFuse-13B, an open-sourced pre-trained code LLM. It is specifically designed for code-related tasks with both English and Chinese prompts. CodeFuse achieves its effectiveness by utilizing a high quality pre-training dataset.
  arXiv  Detail & Related papers  (2023-10-10T02:38:44Z)
• XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations [25.50509874992198]
  Cross-Lingual Semantic Parsing aims to translate queries in multiple natural languages into meaning representations. Existing CLSP models are separately proposed and evaluated on datasets of limited tasks and applications. We present XSemPLR, a unified benchmark for cross-lingual semantic parsing featured with 22 natural languages and 8 meaning representations.
  arXiv  Detail & Related papers  (2023-06-07T01:09:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.