Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selection

• URL: http://arxiv.org/abs/2608.00207v1
• Date: Fri, 31 Jul 2026 18:43:07 GMT
• Title: Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selection
• Abstract summary: Large Language Models (LLMs) perform strongly in English medical tasks but degrade substantially in Arabic.<n>We investigate this assumption via tuned lens probing and causal activation patching.<n>We propose Targeted Low-Rank Adaptation (TLoRA), restricted to the layer window where cross-lingual representations diverge.
• Score: 10.674512408815836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) perform strongly in English medical tasks but degrade substantially in Arabic, a gap widely attributed to limited training data. We systematically investigate this assumption via tuned lens probing and causal activation patching, and find that Arabic medical knowledge is present in intermediate model representations but fails to surface at the output. This mechanistic insight motivates a targeted adaptation strategy: rather than fine-tuning the full network, we propose Targeted Low-Rank Adaptation (TLoRA), restricted to the layer window where cross-lingual representations diverge, upstream of the output layers where the failure manifests. We evaluate TLoRA on multiple-choice medical QA, where our approach outperforms full-network LoRA, zero-shot, and few-shot baselines. We further evaluate it on short-answer generation and multi-turn clinical dialogue, where it performs competitively without the need for task-specific finetuning. We additionally introduce AraClinicDialog, a clinician-constructed Arabic medical dialogue benchmark in MSA with validated variants across four Arabic dialects. Together, these contributions demonstrate that mechanistic diagnosis can serve as a practical guide for targeted adaptation in underrepresented-language medical LLMs.

Related papers

• Can Dialects Be Steered Like Languages? Sparse Neurons and Distributed Directions in Arabic LLMs [60.9510933247844]
  Key challenge in Arabic NLP is the scarcity of dialectal data relative to Modern Standard Arabic (MSA)<n>From an interpretability perspective, this raises a fundamental question: where and how are dialectal features encoded within model internals?<n>This study investigates two complementary inference-time approaches that serve simultaneously as interpretability probes and control mechanisms.
  arXiv  Detail & Related papers  (2026-07-04T16:04:44Z)
• Artificial Intolerance: Stigmatizing Language in Clinical Documentation Skews Large Language Model Decision-Making [13.79732306725696]
  Large Language Models (LLMs) are increasingly deployed in high-stakes domains such as clinical decision support and medical documentation.<n>We investigate whether frontier LLMs inherit and propagate this human bias when processing clinical text.
  arXiv  Detail & Related papers  (2026-05-17T02:28:25Z)
• To Adapt or not to Adapt, Rethinking the Value of Medical Knowledge-Aware Large Language Models [0.7939766248249879]
  Recent studies have shown that domain-adapted large language models (LLMs) do not consistently outperform general-purpose counterparts on standard medical benchmarks.<n>We compare general and clinical LLMs on a diverse set of multiple choice clinical question answering tasks in English and Spanish.
  arXiv  Detail & Related papers  (2026-04-08T09:17:55Z)
• MedAlign: A Synergistic Framework of Multimodal Preference Optimization and Federated Meta-Cognitive Reasoning [52.064286116035134]
  We develop MedAlign, a framework to ensure visually accurate LVLM responses for Medical Visual Question Answering (Med-VQA)<n>We first propose a multimodal Direct Preference Optimization (mDPO) objective to align preference learning with visual context.<n>We then design a Retrieval-Aware Mixture-of-Experts (RA-MoE) architecture that utilizes image and text similarity to route queries to a specialized and context-augmented LVLM.
  arXiv  Detail & Related papers  (2025-10-24T02:11:05Z)
• SwasthLLM: a Unified Cross-Lingual, Multi-Task, and Meta-Learning Zero-Shot Framework for Medical Diagnosis Using Contrastive Representations [0.4077787659104315]
  SwasthLLM is a unified, zero-shot, cross-lingual, and multi-task learning framework for medical diagnosis.<n>It operates effectively across English, Hindi, and Bengali without requiring language-specific fine-tuning.<n>SwasthLLM achieves high diagnostic performance, with a test accuracy of 97.22% and an F1-score of 97.17% in supervised settings.
  arXiv  Detail & Related papers  (2025-09-24T21:20:49Z)
• Applications of Small Language Models in Medical Imaging Classification with a Focus on Prompt Strategies [9.1953139634128]
  This study investigates the performance of small language models (SLMs) in a medical imaging classification task.<n>Using the NIH Chest X-ray dataset, we evaluate multiple SLMs on the task of classifying chest X-ray positions.<n>Our results show that certain SLMs achieve competitive accuracy with well-crafted prompts.
  arXiv  Detail & Related papers  (2025-08-18T21:48:45Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [56.61984030508691]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• Structured Outputs Enable General-Purpose LLMs to be Medical Experts [50.02627258858336]
  Large language models (LLMs) often struggle with open-ended medical questions.<n>We propose a novel approach utilizing structured medical reasoning.<n>Our approach achieves the highest Factuality Score of 85.8, surpassing fine-tuned models.
  arXiv  Detail & Related papers  (2025-03-05T05:24:55Z)
• LlaMADRS: Prompting Large Language Models for Interview-Based Depression Assessment [75.44934940580112]
  This study introduces LlaMADRS, a novel framework leveraging open-source Large Language Models (LLMs) to automate depression severity assessment.<n>We employ a zero-shot prompting strategy with carefully designed cues to guide the model in interpreting and scoring transcribed clinical interviews.<n>Our approach, tested on 236 real-world interviews, demonstrates strong correlations with clinician assessments.
  arXiv  Detail & Related papers  (2025-01-07T08:49:04Z)
• Multi-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs [3.1894617416005855]
  Large language models (LLMs) present a promising solution to automate various ophthalmology procedures.<n>LLMs have demonstrated significantly varied performance across different languages in natural language question-answering tasks.<n>This study introduces the first multilingual ophthalmological question-answering benchmark with manually curated questions parallel across languages.
  arXiv  Detail & Related papers  (2024-12-18T20:18:03Z)
• Mitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding [92.32881381717594]
  We introduce ALternate Contrastive Decoding (ALCD) to solve hallucination issues in medical information extraction tasks. ALCD demonstrates significant improvements in resolving hallucination issues compared to conventional decoding methods.
  arXiv  Detail & Related papers  (2024-10-21T07:19:19Z)
• AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator [69.51568871044454]
  We introduce textbfAI Hospital, a framework simulating dynamic medical interactions between emphDoctor as player and NPCs. This setup allows for realistic assessments of LLMs in clinical scenarios. We develop the Multi-View Medical Evaluation benchmark, utilizing high-quality Chinese medical records and NPCs.
  arXiv  Detail & Related papers  (2024-02-15T06:46:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.