Generalistic or Specific Embeddings, Which is Better? An Empirical Study on Search for Clinical Coding in Non-English Languages

• URL: http://arxiv.org/abs/2605.30529v1
• Date: Thu, 28 May 2026 20:06:43 GMT
• Title: Generalistic or Specific Embeddings, Which is Better? An Empirical Study on Search for Clinical Coding in Non-English Languages
• Abstract summary: We build a two-stage retriever (bi-encoder followed by cross-encoder reranker) on Gemini-generated synthetic data covering English, Spanish, Catalan, Italian, Portuguese and French.<n>The bi-encoder alone matches BioBERT-ST on MRR (0.876 vs. 0.866) and overtakes it on R@3 (0.650 vs. 0.626) and R@5 (0.804 vs. 0.790) without English biomedical pretraining.<n> Adding a cross-encoder reranker lifts aggregate R@5 to 0.822 and dominates on four of five languages (+0.0
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sentence-embedding models for semantic search are overwhelmingly developed and evaluated on English corpora. When applied to clinical retrieval in other languages -- particularly retrieval of ICD-10-CM / CIE-10 codes -- recall degrades in ways often masked by aggregate benchmarks. We study whether large generative language models can serve as data factories to close this gap. We build a two-stage retriever (bi-encoder followed by cross-encoder reranker), fine-tuned from a Spanish biomedical encoder (PlanTL-GOB-ES/bsc-bio-ehr-es) on Gemini-generated synthetic data covering English, Spanish, Catalan, Italian, Portuguese and French, and evaluate against BioBERT-ST and the un-tuned Spanish encoder. The bi-encoder alone matches BioBERT-ST on MRR (0.876 vs. 0.866) and overtakes it on R@3 (0.650 vs. 0.626) and R@5 (0.804 vs. 0.790) without English biomedical pretraining. Adding a cross-encoder reranker lifts aggregate R@5 to 0.822 and dominates on four of five languages (+0.017 Spanish, +0.033 Catalan, +0.018 French, +0.037 Portuguese) at the cost of a small English regression. The trade-off is clinically acceptable: Portuguese reaches R@5 = 0.829 vs. BioBERT-ST's 0.714. Contributions: an open recipe for building domain-specific medical retrievers from LLM-generated data; quantification of the learning gain (MRR 0.755 to 0.876, +15.9% with ~19,500 synthetic pairs); and a characterisation of where gains concentrate by language and rank.

Related papers

• Natural Language Code Retrieval for 1C:Enterprise: An Open Benchmark and Efficient Bi-Encoder [0.0]
  We present an open benchmark of 3,413 real-world, PII-scrubbed query-code pairs.<n>We fine-tune on 784,057 synthetic triplets generated by google/gemma-4-26B-A4B-it from public code repositories.
  arXiv  Detail & Related papers  (2026-08-20T12:24:53Z)
• The Multilingual Curse at the Retrieval Layer: Evidence from Amharic [49.25758237235551]
  Strong zero-shot scores on multilingual benchmarks are often taken as evidence that current encoders transfer reliably across many languages.<n>We argue that this assumption breaks down for underrepresented, morphologically rich languages.<n>We compare zero-shot multilingual retrievers, Amharic-fine-tuned multilingual retrievers, and monolingual Amharic retrievers.
  arXiv  Detail & Related papers  (2026-05-23T12:44:30Z)
• Multilingual Multi-Label Emotion Classification at Scale with Synthetic Data [2.3424047967193826]
  We construct a large-scale synthetic training corpus of over 1M multi-label samples across 23 languages.<n>We train and compare six multilingual transformer encoders, from DistilBERT (135M parameters) to XLM-R-Large (560M parameters)<n>To validate against human-annotated data, we evaluate all models zero-shot on GoEmotions (English) and SemEval-2018 Task 1 E-c (English, Arabic, Spanish)
  arXiv  Detail & Related papers  (2026-04-14T12:04:17Z)
• Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech [61.759910921200834]
  Cross-lingual sentence encoders typically cover only a few hundred languages.<n>We introduce OmniSONAR, a new family of omnilingual, cross-lingual and cross-language sentence embedding models.
  arXiv  Detail & Related papers  (2026-03-17T14:47:35Z)
• TurkColBERT: A Benchmark of Dense and Late-Interaction Models for Turkish Information Retrieval [0.0]
  We introduce TurkColBERT, the first benchmark comparing dense encoders and late-interaction models for Turkish retrieval.<n>Our two-stage adaptation pipeline fine-tunes English and multilingual encoders on Turkish NLI/STS tasks, then converts them into ColBERT-style retrievers.<n>We evaluate 10 models across five Turkish BEIR datasets covering scientific, financial, and argumentative domains.
  arXiv  Detail & Related papers  (2025-11-20T16:42:21Z)
• Large Language Models as Medical Codes Selectors: a benchmark using the International Classification of Primary Care [4.633036227475212]
  This study assesses the potential of large language models to assign ICPC-2 codes using a domain-specific search engine.<n>A dataset of 437 Brazilian Portuguese clinical expressions, each annotated with ICPC-2 codes, was used.
  arXiv  Detail & Related papers  (2025-07-19T16:11:10Z)
• Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval [49.1574468325115]
  We introduce Amharic-specific dense retrieval models based on pre-trained Amharic BERT and RoBERTa backbones.<n>Our proposed RoBERTa-Base-Amharic-Embed model (110M parameters) achieves a 17.6% relative improvement in MRR@10.<n>More compact variants, such as RoBERTa-Medium-Amharic-Embed (42M) remain competitive while being over 13x smaller.
  arXiv  Detail & Related papers  (2025-05-25T23:06:20Z)
• SCRum-9: Multilingual Stance Classification over Rumours on Social Media [15.412870757706473]
  SCRum-9 is the largest dataset for Rumour analysis in 9 languages, containing 7,516 tweets from X.<n>This paper innovates by exploring the use of multilingual synthetic data for stance classification.<n> SCRum-9 is publicly released to the research community with potential to foster further research on multilingual analysis of misleading narratives on social media.
  arXiv  Detail & Related papers  (2025-05-25T00:50:43Z)
• CLaCLab at SocialDisNER: Using Medical Gazetteers for Named-Entity Recognition of Disease Mentions in Spanish Tweets [0.7918886297003019]
  This paper summarizes the CLaC submission for SMM4H 2022 Task 10 which concerns the recognition of diseases mentioned in Spanish tweets. We encode each token with a transformer encoder using features from Multilingual RoBERTa Large, UMLS gazetteer, and DISTEMIST gazetteer. We obtain a strict F1 score of 0.869, with competition mean of 0.675, standard deviation of 0.245, and median of 0.761.
  arXiv  Detail & Related papers  (2022-09-08T02:08:51Z)
• Few-Shot Cross-lingual Transfer for Coarse-grained De-identification of Code-Mixed Clinical Texts [56.72488923420374]
  Pre-trained language models (LMs) have shown great potential for cross-lingual transfer in low-resource settings. We show the few-shot cross-lingual transfer property of LMs for named recognition (NER) and apply it to solve a low-resource and real-world challenge of code-mixed (Spanish-Catalan) clinical notes de-identification in the stroke.
  arXiv  Detail & Related papers  (2022-04-10T21:46:52Z)
• CoSQA: 20,000+ Web Queries for Code Search and Question Answering [63.92224685262063]
  CoSQA dataset includes 20,604 labels for pairs of natural language queries and codes. We introduce a contrastive learning method dubbed CoCLR to enhance query-code matching. We show that evaluated on CodeXGLUE with the same CodeBERT model, training on CoSQA improves the accuracy of code question answering by 5.1%.
  arXiv  Detail & Related papers  (2021-05-27T15:37:21Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.