Clinically Structured Rank-Gated LoRA for Cross-Benchmark Medical Question Answering

• URL: http://arxiv.org/abs/2606.31432v3
• Date: Thu, 02 Jul 2026 08:43:43 GMT
• Title: Clinically Structured Rank-Gated LoRA for Cross-Benchmark Medical Question Answering
• Abstract summary: BiRG-LoRA is a single-adapter rank-gated LoRA method for medical question answering.<n>BiRG-LoRA achieves the highest four-benchmark macro-average accuracy among trainable PEFT baselines.
• Score: 4.896659944013307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Medical multiple-choice question answering requires parameter-efficient adaptation across heterogeneous knowledge domains and reasoning operations. A medication question, a diagnostic decision, a public-health item, and a nursing-action item may require different low-rank updates, while some recall items should preserve the base model's representation with only mild adapter intervention. We propose BiRG-LoRA, a single-adapter rank-gated LoRA method for medical question answering. BiRG-LoRA keeps one LoRA module per target layer but makes its rank dimension input-conditioned: for each question, a biaxial gate combines hidden semantic evidence with specialty/profession priors, clinical-operation priors, and their interaction to select a sparse top-$k$ subset of rank atoms. A scalar injection coefficient further controls the strength of the selected adapter update. Under a matched Qwen3-8B CMB-source protocol, BiRG-LoRA achieves the highest four-benchmark macro-average accuracy among trainable PEFT baselines and matched routing controls: 69.31% averaged over CMB, CMExam, MedQA, and MedMCQA. It improves over MoELoRA by 0.89 percentage points while using 28.1% fewer trainable parameters; a paired, benchmark-stratified bootstrap over final predictions gives a 95% confidence interval of [0.42, 1.37] for this macro-average gain. Basic controls show that BiRG-LoRA also improves over vanilla LoRA r16 and active-rank-matched LoRA r4 by 0.83 macro points, and an evaluation-time weak-axis perturbation check suggests that performance is not brittle to moderate tag noise. The results support a bounded claim: clinically structured rank allocation improves cross-benchmark medical QA under a matched single-seed protocol, while training-seed variance remains future work.

Related papers

• Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QA [12.797519888387628]
  We propose root-mean-square Robustness-based Sample Prioritization (RMS-RSP) to perturb hidden states at rationale tokens.<n> RMS-RSP improves robust accuracy and semantic consistency by 1.91 and 2.85 points on average, with the same direction on all five datasets.
  arXiv  Detail & Related papers  (2026-09-09T03:58:59Z)
• Statistical Inference for Rank Allocation in Low-Rank Adaptation [54.886931928255564]
  Low-rank adaptation (LoRA) has become a widely used parameter-efficient fine-tuning method for large language models.<n>We propose StatLoRA, a statistical inference-based rank allocation method.<n>We show that StatLoRA achieves comparable or better performance than vanilla LoRA, AdaLoRA, and I-LoRA under matched rank budgets.
  arXiv  Detail & Related papers  (2026-07-22T14:25:59Z)
• TriageRA-CCF: Source-Side Clinical Confidence and Coverage Signals for Adaptive Rank Budgeting in Medical LLMs [8.601855281320903]
  TriageRA-CCF is a source-side teacher for adaptive rank-budgeted medical large language models.<n>It combines three signals computed only from source training data: base-model answer confidence, metadata-cell clinical coverage, and a counterfactual close-miss proxy.<n>Under a matched CMB-source training protocol, TriageRA-CCF achieves the best average accuracy among LoRA, DoRA, and MoELoRA baselines.
  arXiv  Detail & Related papers  (2026-06-28T12:52:07Z)
• iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis [15.311128089548573]
  iLoRA is the first Bayesian graph-conditioned LoRA framework.<n>It infers a latent interaction graph from the input and uses it to generate input-conditioned LoRA updates.<n>We instantiate this idea for microbiome diagnosis, where disease state can depend on both species-level abundance and microbe-microbe cross-talk.
  arXiv  Detail & Related papers  (2026-05-28T16:26:06Z)
• SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning [57.85676271833619]
  Low-rank Adaptation (LoRA) uses a low-rank update method to simulate full parameter fine-tuning.<n>As the rank increases, more principal singular directions are preserved, which generally improves the model's performance.<n>We propose SMoA, a textbfSpectrum textbfModulation textbfAdapter that enlarges the accessible family of spectrum-aware updates.
  arXiv  Detail & Related papers  (2026-05-20T13:19:28Z)
• FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation [0.0]
  Low-rank adaptation (LoRA) assigns a uniform rank to every adapted weight matrix.<n>We compute the gradient variance of each LoRA-B matrix as a proxy for layer informativeness, and redistribute the rank budget proportionally.<n>The resulting adapter is a standard LoRA with a per-layer rank pattern.
  arXiv  Detail & Related papers  (2026-05-16T04:03:21Z)
• A Multi-Agent Framework for Medical AI: Leveraging Fine-Tuned GPT, LLaMA, and DeepSeek R1 for Evidence-Based and Bias-Aware Clinical Query Processing [0.4349324020366305]
  Large language models (LLMs) show promise for healthcare question answering, but clinical use is limited by weak verification, insufficient evidence grounding, and unreliable confidence signalling.<n>We propose a multi-agent medical QA framework that combines complementary LLMs with evidence retrieval, uncertainty estimation, and bias checks to improve answer reliability.
  arXiv  Detail & Related papers  (2026-02-15T14:17:27Z)
• From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG [65.0203623486525]
  Large Language Models (LLMs) exhibit high reasoning capacity in medical question-answering.<n>Their tendency to produce hallucinations and outdated knowledge poses critical risks in healthcare fields.<n>Existing methods rely on noisy token-level signals and lack the multi-round refinement required for complex reasoning.
  arXiv  Detail & Related papers  (2026-02-06T08:25:30Z)
• P-RAG: Prompt-Enhanced Parametric RAG with LoRA and Selective CoT for Biomedical and Multi-Hop QA [9.399056753263757]
  Retrieval-Augmented Generation (RAG) addresses this constraint by retrieving external knowledge during inference.<n>We evaluate three RAG variants-Standard RAG, DA-RAG, and our proposed Prompt-Enhanced Parametric RAG (P-RAG)<n>P-RAG integrates parametric knowledge within the LLM and retrieved evidence, guided by Chain-of-Thought (CoT) prompting and Low-Rank Adaptation (LoRA)
  arXiv  Detail & Related papers  (2026-02-02T03:42:45Z)
• Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update [50.36542772932594]
  Low-Rank Adaptation (LoRA) fine-tunes large models by learning low-rank updates on top of frozen weights.<n>There is still a gap between full training with low-rank projections (SVDLoRA) and LoRA fine-tuning, indicating that LoRA steps can be further improved.
  arXiv  Detail & Related papers  (2025-09-24T10:32:50Z)
• Automatic Rank Determination for Low-Rank Adaptation via Submodular Function Maximization [56.78271181959529]
  SubLoRA is a rank determination method for Low-Rank Adaptation (LoRA) based on submodular function.<n>Our method combines solid theoretical foundations, second-order accuracy, and practical computational efficiency.
  arXiv  Detail & Related papers  (2025-07-02T15:56:40Z)
• LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimization [78.93425154518705]
  Low-rank adaption (LoRA) is a widely used parameter-efficient finetuning method for LLM that reduces memory requirements.<n>This paper introduces LoRA-RITE, a novel adaptive matrix preconditioning method for LoRA optimization.
  arXiv  Detail & Related papers  (2024-10-27T22:57:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.