F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World

• URL: http://arxiv.org/abs/2603.19223v1
• Date: Thu, 19 Mar 2026 17:59:21 GMT
• Title: F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World
• Abstract summary: F2LLM-v2 is a new family of general-purpose, multilingual embedding models in 8 distinct sizes ranging from 80M to 14B.<n>Trained on a newly curated composite of 60 million publicly available high-quality data samples, F2LLM-v2 supports more than 200 languages.
• Score: 19.088644745246373
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present F2LLM-v2, a new family of general-purpose, multilingual embedding models in 8 distinct sizes ranging from 80M to 14B. Trained on a newly curated composite of 60 million publicly available high-quality data samples, F2LLM-v2 supports more than 200 languages, with a particular emphasis on previously underserved mid- and low-resource languages. By integrating a two-stage LLM-based embedding training pipeline with matryoshka learning, model pruning, and knowledge distillation techniques, we present models that are far more efficient than previous LLM-based embedding models while retaining competitive performances. Extensive evaluations confirm that F2LLM-v2-14B ranks first on 11 MTEB benchmarks, while the smaller models in the family also set a new state of the art for resource-constrained applications. To facilitate open-source embedding model research, we release all models, data, code, and intermediate checkpoints.

Related papers

• MiniLingua: A Small Open-Source LLM for European Languages [47.78284815754002]
  MiniLingua is a multilingual open-source LLM of one billion parameters trained from scratch for 13 European languages.<n>We release model weights, tokenizer and source code used for data processing and model training.
  arXiv  Detail & Related papers  (2025-12-15T13:12:42Z)
• Llama-Embed-Nemotron-8B: A Universal Text Embedding Model for Multilingual and Cross-Lingual Tasks [4.739000717606982]
  llama-embed-nemotron-8b is an open-weights text embedding model.<n>It achieves state-of-the-art performance on the Multilingual Massive Text Embedding Benchmark.
  arXiv  Detail & Related papers  (2025-11-10T12:13:16Z)
• F2LLM Technical Report: Matching SOTA Embedding Performance with 6 Million Open-Source Data [19.088644745246373]
  We introduce F2LLM, a suite of state-of-the-art embedding models in three sizes: 0.6B, 1.7B, and 4B.<n>F2LLM is directly finetuned from foundation models on 6 million query-document-negatives curated from open-source, non-synthetic datasets.<n>On the MTEB English leaderboard, F2LLM-4B ranks 2nd among models with approximately 4B parameters and 7th overall, while F2LLM-1.7B ranks 1st among models in the 1B-2B size range.
  arXiv  Detail & Related papers  (2025-10-02T17:58:49Z)
• Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study [13.409987421121405]
  GemmaX2-28 is a 9B model achieving top-tier multilingual translation performance across 28 languages.<n>GemmaX2-28 consistently outperforms the state-of-the-art (SOTA) models such as TowerInstruct and XALMA.
  arXiv  Detail & Related papers  (2025-02-04T16:57:03Z)
• KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model [27.25688303240741]
  KaLM-Embedding is a general multilingual embedding model that leverages a large quantity of cleaner, more diverse, and domain-specific training data.<n>Our model has been trained with key techniques proven to enhance performance.
  arXiv  Detail & Related papers  (2025-01-02T03:17:51Z)
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling [191.7830199016589]
  We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0.<n>InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet.<n>We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems.
  arXiv  Detail & Related papers  (2024-12-06T18:57:08Z)
• VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks [60.5257456681402]
  We study the potential for building universal embeddings capable of handling a wide range of downstream tasks.<n>We build a series of VLM2Vec models on SoTA VLMs like Phi-3.5-V, LLaVA-1.6 and evaluate them on MMEB's evaluation split.<n>Our results show that VLM2Vec achieves an absolute average improvement of 10% to 20% over existing multimodal embedding models.
  arXiv  Detail & Related papers  (2024-10-07T16:14:05Z)
• LLAVADI: What Matters For Multimodal Large Language Models Distillation [77.73964744238519]
  In this work, we do not propose a new efficient model structure or train small-scale MLLMs from scratch. Our studies involve training strategies, model choices, and distillation algorithms in the knowledge distillation process. By evaluating different benchmarks and proper strategy, even a 2.7B small-scale model can perform on par with larger models with 7B or 13B parameters.
  arXiv  Detail & Related papers  (2024-07-28T06:10:47Z)
• Unlocking the Potential of Model Merging for Low-Resource Languages [66.7716891808697]
  Adapting large language models to new languages typically involves continual pre-training (CT) followed by supervised fine-tuning (SFT) We propose model merging as an alternative for low-resource languages, combining models with distinct capabilities into a single model without additional training. Experiments based on Llama-2-7B demonstrate that model merging effectively endows LLMs for low-resource languages with task-solving abilities, outperforming CT-then-SFT in scenarios with extremely scarce data.
  arXiv  Detail & Related papers  (2024-07-04T15:14:17Z)
• Tele-FLM Technical Report [96.19923831660266]
  We introduce Tele-FLM (aka FLM-2), a 52B open-sourced multilingual large language model. It features a stable, efficient pre-training paradigm and enhanced factual judgment capabilities. It is comparable to strong open-sourced models that involve larger pre-training FLOPs, such as Llama2-70B and DeepSeek-67B.
  arXiv  Detail & Related papers  (2024-04-25T14:34:47Z)
• YAYI 2: Multilingual Open-Source Large Language Models [53.92832054643197]
  We propose YAYI 2, including both base and chat models, with 30 billion parameters. YAYI 2 is pre-trained from scratch on a multilingual corpus which contains 2.65 trillion tokens filtered by our pre-training data processing pipeline. The base model is aligned with human values through supervised fine-tuning with millions of instructions and reinforcement learning from human feedback.
  arXiv  Detail & Related papers  (2023-12-22T17:34:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.