Querit-Reranker: Training Compact Multilingual Rerankers via Efficient Label-Free Distribution Adaptation

• URL: http://arxiv.org/abs/2606.19037v1
• Date: Wed, 17 Jun 2026 13:06:47 GMT
• Title: Querit-Reranker: Training Compact Multilingual Rerankers via Efficient Label-Free Distribution Adaptation
• Abstract summary: We present Querit-Reranker, a family of multilingual cross-encoder rerankers trained with a data-centric pipeline.<n>Our pipeline first learns general relevance modeling from large-scale ranking-oriented data, then adapts to target distributions.<n>Querit-Reranker-A0.4B improves average nDCG@10 from 54.11 to 59.28 on BEIR and from 59.87 to 67.70 on MIRACL.
• Score: 43.66118282567413
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deployable multilingual rerankers must generalize across languages, domains, and target ranking tasks while remaining efficient enough for second-stage reranking. However, adapting them to new target distributions typically requires extensive task-specific relevance annotations, which are costly to obtain. We present Querit-Reranker, a family of multilingual cross-encoder rerankers trained with a data-centric pipeline for label-efficient adaptation. We instantiate it as Querit-Reranker-A0.4B, initialized from an in-house MoE backbone with 0.4B activated parameters, and Querit-Reranker-4B, initialized from Qwen3-Embedding-4B. Our pipeline first learns general relevance modeling from large-scale ranking-oriented data, then adapts to target distributions through synthetic-query mining with teacher scores as continuous soft labels. To consolidate complementary task-adapted strengths, we further merge checkpoints via spherical linear interpolation, obtaining a single deployable model without runtime ensembling overhead. Using Qwen3-Embedding-0.6B as the shared first-stage retriever, Querit-Reranker-A0.4B improves average nDCG@10 from 54.11 to 59.28 on BEIR and from 59.87 to 67.70 on MIRACL. On MTEB Multilingual v2 Reranking, it also substantially outperforms larger embedding-based baselines, while Querit-Reranker-4B further achieves state-of-the-art performance among publicly available models. We release both models on Hugging Face.

Related papers

• Models for minimalist RAG: B1ade 335M Embedding and 1B Parameter Small Language Models [2.1665689529884697]
  We present B1ade, an efficient RAG architecture comprising two purpose-built components: a compact embedding model and a purpose-built SLM.<n>B1ade-embed, a 335M parameter retrieval model constructed via parameter-free fusion of five pretrained encoders achieves top MTEB scores among sub-500M models with zero additional training.<n>B1ade-1B cites retrieved passages in 42.4% of responses, exceeding the attribution rate of its training distribution by 5.5 percentage points.
  arXiv  Detail & Related papers  (2026-07-29T22:41:25Z)
• RelPrism: A Multi-Faceted Pre-training Framework with Self-Generated Tasks for Relational Databases [49.47033473074821]
  We propose RelPrism, a multi-faceted self-supervised learning framework for RDBs.<n>RDB tasks often require multi-faceted information across different perspectives and granularities.<n>Our experiments show that RelPrism improves ROC-AUC by 4.15% for classification and reduces MAE by 10.75% for regression over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-05-22T05:19:58Z)
• EmbGen: Teaching with Reassembled Corpora [0.300988853836121]
  We introduce EmbGen, a synthetic data generation pipeline that decomposes a corpus into entity-description pairs.<n>We evaluate EmbGen against EntiGraph, InstructLab and Knowledge-Instruct on three datasets of varied semantic heterogeneity.
  arXiv  Detail & Related papers  (2026-05-19T05:40:12Z)
• Bayesian Model Merging [17.887004278413915]
  Model merging aims to combine multiple task-specific expert models into a single model without joint retraining.<n>This paper introduces Bayesian Model Merging (BMM), a plug-and-play bi-level optimization framework.<n>BMM consistently outperforms all plug-and-play anchor baselines.
  arXiv  Detail & Related papers  (2026-05-13T00:36:47Z)
• Peer-Predictive Self-Training for Language Model Reasoning [15.499995918997756]
  We propose a label-free fine-tuning framework in which multiple language models improve collaboratively.<n>Given a prompt question, the models generate responses sequentially; the final aggregated answer serves as an internal target for learning.<n>On mathematical reasoning benchmarks, PST improves exact-match accuracy by 2.2 to 4.3 percentage points across Gemma-2-2B, LLaMA-3.2-1B, and Qwen-2.5-1.5B.
  arXiv  Detail & Related papers  (2026-04-14T23:29:44Z)
• Bagging-Based Model Merging for Robust General Text Embeddings [73.51674133699196]
  General-purpose text embedding models underpin a wide range of NLP and information retrieval applications.<n>We present a systematic study of multi-task training for text embeddings from two perspectives: data scheduling and model merging.<n>We propose Bagging-based rObust mOdel Merging (BOOM), which trains multiple embedding models on sampled subsets and merges them into a single model.
  arXiv  Detail & Related papers  (2026-02-05T15:45:08Z)
• Limited Reference, Reliable Generation: A Two-Component Framework for Tabular Data Generation in Low-Data Regimes [7.036974567001374]
  ReFine is a framework that guides generation toward domain-specific feature distribution.<n>Experiments on various regression and classification benchmarks demonstrate that ReFine consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-09-12T04:34:46Z)
• Finding Needles in Emb(a)dding Haystacks: Legal Document Retrieval via Bagging and SVR Ensembles [51.0691253204425]
  We introduce a retrieval approach leveraging Support Vector Regression ensembles, bootstrap aggregation (bagging), and embedding spaces on the German dataset for Legal Information Retrieval (GerDaLIR)<n>We show improved recall over the baselines using our voting ensemble, suggesting promising initial results, without training or fine-tuning any deep learning models.
  arXiv  Detail & Related papers  (2025-01-09T07:21:44Z)
• Towards Generalizable Trajectory Prediction Using Dual-Level Representation Learning And Adaptive Prompting [107.4034346788744]
  Existing vehicle trajectory prediction models struggle with generalizability, prediction uncertainties, and handling complex interactions.<n>We propose Perceiver with Register queries (PerReg+), a novel trajectory prediction framework that introduces: (1) Dual-Level Representation Learning via Self-Distillation (SD) and Masked Reconstruction (MR), capturing global context and fine-grained details; (2) Enhanced Multimodality using register-based queries and pretraining, eliminating the need for clustering and suppression; and (3) Adaptive Prompt Tuning during fine-tuning, freezing the main architecture and optimizing a small number of prompts for efficient adaptation.
  arXiv  Detail & Related papers  (2025-01-08T20:11:09Z)
• X2Parser: Cross-Lingual and Cross-Domain Framework for Task-Oriented Compositional Semantic Parsing [51.81533991497547]
  Task-oriented compositional semantic parsing (TCSP) handles complex nested user queries. We present X2 compared a transferable Cross-lingual and Cross-domain for TCSP. We propose to predict flattened intents and slots representations separately and cast both prediction tasks into sequence labeling problems.
  arXiv  Detail & Related papers  (2021-06-07T16:40:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.