Severity-Aware Weighted Loss for Arabic Medical Text Generation

• URL: http://arxiv.org/abs/2604.06346v1
• Date: Tue, 07 Apr 2026 18:26:41 GMT
• Title: Severity-Aware Weighted Loss for Arabic Medical Text Generation
• Abstract summary: We propose a severity-aware weighted loss for fine-tuning Arabic language models on medical complaint-response data.<n>Experiments are conducted using the MAQA dataset, which provides Arabic medical complaints and trusted human responses.
• Score: 0.25489046505746704
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Large language models have shown strong potential for Arabic medical text generation; however, traditional fine-tuning objectives treat all medical cases uniformly, ignoring differences in clinical severity. This limitation is particularly critical in healthcare settings, where errors in severe cases contain higher clinical risk. In this work, we propose a severity-aware weighted loss for fine-tuning Arabic language models on medical complaint-response data. The method depends on soft severity probabilities to dynamically scale token-level loss contributions during optimization, thereby prioritizing clinically critical interactions without modifying model architectures. Experiments are conducted using the MAQA dataset, which provides Arabic medical complaints and trusted human responses. Severity labels and probabilistic scores are automatically derived using a fine-tuned AraBERT-based classifier and incorporated exclusively at the loss level. The proposed approach is evaluated across ten Arabic large language models of varying architectures and parameter scales. While standard cross-entropy fine-tuning yields only modest improvements, severity-aware optimization consistently achieves larger gains. Using a balanced weighting configuration, performance improves from 54.04% to 66.14% for AraGPT2-Base, from 59.16% to 67.18% for AraGPT2-Medium, and from 57.83% to 66.86% for Qwen2.5-0.5B, with peak performance reaching 67.18%. Overall, severity-aware fine-tuning delivers improvements of up to 12.10% over non-fine-tuned baselines, demonstrating robust and architecture-consistent gains.

Related papers

• What Makes Deep Learning Work for Traditional Chinese Medicine Tongue Diagnosis? A Comprehensive Ablation Study [6.004599301889361]
  We conducted a systematic ablation study spanning 20+ model versions on TongueDx2 (5,109 images, 976 expert-annotated) and a merged dataset of 11,101 samples.<n>Best 976-sample model achieved weighted-F1 of 0.6625 using ConvNeXt-Tiny with restrained augmentation and weak-group ensemble.<n>Six key design principles emerged: ConvNeXt-Tiny offers optimal parameter efficiency; BCE substantially outperforms Asymmetric Loss; restrained color augmentation is critical; weak-group ensemble replacement is critical.
  arXiv  Detail & Related papers  (2026-07-30T12:55:05Z)
• MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection [0.7039739305668745]
  This study proposes a two-phase framework for Arabic mental health text classification.<n>In phase 1, three Arabic pre-trained language models undergo Domain-Adaptive and Task-Adaptive Pretraining.<n>In phase 2, the selected model is assessed across four configurations with full fine-tuning and Low-Rank Adaptation.
  arXiv  Detail & Related papers  (2026-06-10T20:20:41Z)
• Enhancing ASR Performance in the Medical Domain for Dravidian Languages [4.188779570200089]
  This work proposes a novel confidence-aware training framework that integrates real and synthetic speech data.<n>A 5-gram KenLM language model is applied for post-decoding correction.<n>Results show that the hybrid confidence-aware approach with learnable weights substantially reduces recognition errors.
  arXiv  Detail & Related papers  (2026-04-10T09:41:26Z)
• Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech [69.86604856129883]
  Dysarthric speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive speech technologies.<n>We propose a three-stage framework that leverages unlabeled dysarthric speech and large-scale typical speech datasets to scale training.
  arXiv  Detail & Related papers  (2026-03-16T23:00:07Z)
• PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncology [48.732366302949515]
  Large language models (LLMs) have achieved expert-level performance on standardized examinations, yet multiple-choice accuracy poorly reflects real-world clinical utility and safety.<n>We developed a human-in-the-loop pipeline to create expert rubrics for de-identified patient questions.<n>We evaluated 22 proprietary and open-source LLMs using an LLM-as-a-judge framework, measuring clinical completeness, factual accuracy, and web-search integration.
  arXiv  Detail & Related papers  (2026-03-02T00:50:39Z)
• A DeepSeek-Powered AI System for Automated Chest Radiograph Interpretation in Clinical Practice [83.11942224668127]
  Janus-Pro-CXR (1B) is a chest X-ray interpretation system based on DeepSeek Janus-Pro model.<n>Our system outperforms state-of-the-art X-ray report generation models in automated report generation.
  arXiv  Detail & Related papers  (2025-12-23T13:26:13Z)
• Orthographic Constraint Satisfaction and Human Difficulty Alignment in Large Language Models [2.3991974633684854]
  Large language models must satisfy hard orthographic constraints during controlled text generation.<n>We evaluate 28 configurations spanning three model families on 58 word puzzles requiring character-level constraint satisfaction.
  arXiv  Detail & Related papers  (2025-11-26T06:12:33Z)
• Arabic Large Language Models for Medical Text Generation [0.5483130283061118]
  This study proposes an approach that fine-tunes large language models (LLMs) for Arabic medical text generation.<n>The system is designed to assist patients by providing accurate medical advice, diagnoses, drug recommendations, and treatment plans based on user input.
  arXiv  Detail & Related papers  (2025-09-12T09:37:26Z)
• Enhancing Clinical Text Classification via Fine-Tuned DRAGON Longformer Models [7.514574388197471]
  This study explores the optimization of the DRAGON Longformer base model for clinical text classification.<n>A dataset of 500 clinical cases containing structured medical observations was used.<n>The optimized model achieved notable performance gains.
  arXiv  Detail & Related papers  (2025-07-13T03:10:19Z)
• Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval [49.1574468325115]
  We introduce Amharic-specific dense retrieval models based on pre-trained Amharic BERT and RoBERTa backbones.<n>Our proposed RoBERTa-Base-Amharic-Embed model (110M parameters) achieves a 17.6% relative improvement in MRR@10.<n>More compact variants, such as RoBERTa-Medium-Amharic-Embed (42M) remain competitive while being over 13x smaller.
  arXiv  Detail & Related papers  (2025-05-25T23:06:20Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Large Language Models to Identify Social Determinants of Health in Electronic Health Records [2.168737004368243]
  Social determinants of health (SDoH) have an important impact on patient outcomes but are incompletely collected from the electronic health records (EHRs) This study researched the ability of large language models to extract SDoH from free text in EHRs, where they are most commonly documented. 800 patient notes were annotated for SDoH categories, and several transformer-based models were evaluated.
  arXiv  Detail & Related papers  (2023-08-11T19:18:35Z)
• Generative models improve fairness of medical classifiers under distribution shifts [49.10233060774818]
  We show that learning realistic augmentations automatically from data is possible in a label-efficient manner using generative models. We demonstrate that these learned augmentations can surpass ones by making models more robust and statistically fair in- and out-of-distribution.
  arXiv  Detail & Related papers  (2023-04-18T18:15:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.