EuroLLM-9B: Technical Report

• URL: http://arxiv.org/abs/2506.04079v2
• Date: Mon, 16 Jun 2025 18:23:31 GMT
• Title: EuroLLM-9B: Technical Report
• Authors: Pedro Henrique Martins, João Alves, Patrick Fernandes, Nuno M. Guerreiro, Ricardo Rei, Amin Farajian, Mateusz Klimaszewski, Duarte M. Alves, José Pombal, Nicolas Boizard, Manuel Faysse, Pierre Colombo, François Yvon, Barry Haddow, José G. C. de Souza, Alexandra Birch, André F. T. Martins,
• Abstract summary: EuroLLM-9B is a large language model trained from scratch to cover all 24 official European Union languages and 11 additional languages.<n>We provide a comprehensive overview of EuroLLM-9B's development, including tokenizer design, architectural specifications, data filtering, and training procedures.
• Score: 79.96096140260924
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This report presents EuroLLM-9B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11 additional languages. EuroLLM addresses the issue of European languages being underrepresented and underserved in existing open large language models. We provide a comprehensive overview of EuroLLM-9B's development, including tokenizer design, architectural specifications, data filtering, and training procedures. We describe the pre-training data collection and filtering pipeline, including the creation of EuroFilter, an AI-based multilingual filter, as well as the design of EuroBlocks-Synthetic, a novel synthetic dataset for post-training that enhances language coverage for European languages. Evaluation results demonstrate EuroLLM-9B's competitive performance on multilingual benchmarks and machine translation tasks, establishing it as the leading open European-made LLM of its size. To support open research and adoption, we release all major components of this work, including the base and instruction-tuned models, the EuroFilter classifier, and the synthetic post-training dataset.

Related papers

• Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters [53.59868121093848]
  We introduce Seed-X, a family of open-source language models (LLMs) with 7B parameter size.<n>The base model is pre-trained on a diverse, high-quality dataset encompassing both monolingual and bilingual content across 28 languages.<n>The instruct model is then finetuned to translate by Chain-of-Thought (CoT) reasoning and further enhanced through reinforcement learning (RL) to achieve better generalization across diverse language pairs.
  arXiv  Detail & Related papers  (2025-07-18T03:19:43Z)
• Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs [29.595342315049106]
  We present two multilingual LLMs designed to embrace Europe's linguistic diversity by supporting all 24 official languages of the European Union. We detail the models' development principles, i.e., data composition, tokenizer optimization, and training methodologies.
  arXiv  Detail & Related papers  (2024-09-30T16:05:38Z)
• EuroLLM: Multilingual Language Models for Europe [76.89545643715368]
  We introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs. We outline the progress made to date, detailing our data collection and filtering process. We report our performance on multilingual general benchmarks and machine translation.
  arXiv  Detail & Related papers  (2024-09-24T16:51:36Z)
• Datasets for Multilingual Answer Sentence Selection [59.28492975191415]
  We introduce new high-quality datasets for AS2 in five European languages (French, German, Italian, Portuguese, and Spanish) Results indicate that our datasets are pivotal in producing robust and powerful multilingual AS2 models.
  arXiv  Detail & Related papers  (2024-06-14T16:50:29Z)
• D\'olares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English [67.48541936784501]
  Tois'on de Oro is the first framework that establishes instruction datasets, finetuned LLMs, and evaluation benchmark for financial LLMs in Spanish joint with English. We construct a rigorously curated bilingual instruction dataset including over 144K Spanish and English samples from 15 datasets covering 7 tasks. We evaluate our model and existing LLMs using FLARE-ES, the first comprehensive bilingual evaluation benchmark with 21 datasets covering 9 tasks.
  arXiv  Detail & Related papers  (2024-02-12T04:50:31Z)
• Extrapolating Large Language Models to Non-English by Aligning Languages [109.09051737966178]
  Existing large language models show disparate capability across different languages. In this paper, we empower pre-trained LLMs on non-English languages by building semantic alignment across languages.
  arXiv  Detail & Related papers  (2023-08-09T13:32:06Z)
• PyEuroVoc: A Tool for Multilingual Legal Document Classification with EuroVoc Descriptors [0.3007949058551534]
  We propose a unified framework for EuroVoc classification on 22 languages by fine-tuning modern Transformer-based pretrained language models. The code and the fine-tuned models were open sourced, together with a programmatic interface that eases the process of loading the weights of a trained model and of classifying a new document.
  arXiv  Detail & Related papers  (2021-08-02T19:46:21Z)
• Natural Language Processing Chains Inside a Cross-lingual Event-Centric Knowledge Pipeline for European Union Under-resourced Languages [0.0]
  This article presents the strategy for developing a platform containing Language Processing Chains for European Union languages. These chains are part of the first step of an event-centric knowledge processing pipeline whose aim is to process multilingual media information about major events that can cause an impactin Europe and the rest of the world.
  arXiv  Detail & Related papers  (2020-10-23T14:26:30Z)
• Beyond English-Centric Multilingual Machine Translation [74.21727842163068]
  We create a true Many-to-Many multilingual translation model that can translate directly between any pair of 100 languages. We build and open source a training dataset that covers thousands of language directions with supervised data, created through large-scale mining. Our focus on non-English-Centric models brings gains of more than 10 BLEU when directly translating between non-English directions while performing competitively to the best single systems of WMT.
  arXiv  Detail & Related papers  (2020-10-21T17:01:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.