Eliciting Medical Reasoning with Knowledge-enhanced Data Synthesis: A Semi-Supervised Reinforcement Learning Approach

• URL: http://arxiv.org/abs/2604.11547v1
• Date: Mon, 13 Apr 2026 14:37:38 GMT
• Title: Eliciting Medical Reasoning with Knowledge-enhanced Data Synthesis: A Semi-Supervised Reinforcement Learning Approach
• Abstract summary: Existing approaches distill chain-of-thought reasoning traces from large proprietary models via supervised fine-tuning, then conduct reinforcement learning (RL)<n>We propose MedSSR, a Medical Knowledge-enhanced data Synthesis and Semi-supervised Reinforcement learning framework.<n>Our framework first employs rare disease knowledge to synthesize distribution-controllable reasoning questions.<n>We then utilize the policy model itself to generate high-quality pseudo-labels. This enables a two-stage, intrinsic-to-extrinsic training paradigm: self-supervised RL on the pseudo-labeled data, followed by supervised RL on the human-annotated real data
• Score: 62.0906177191353
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: While large language models hold promise for complex medical applications, their development is hindered by the scarcity of high-quality reasoning data. To address this issue, existing approaches typically distill chain-of-thought reasoning traces from large proprietary models via supervised fine-tuning, then conduct reinforcement learning (RL). These methods exhibit limited improvement on underrepresented domains like rare diseases while incurring substantial costs from generating complex reasoning chains. To efficiently enhance medical reasoning, we propose MedSSR, a Medical Knowledge-enhanced data Synthesis and Semi-supervised Reinforcement learning framework. Our framework first employs rare disease knowledge to synthesize distribution-controllable reasoning questions. We then utilize the policy model itself to generate high-quality pseudo-labels. This enables a two-stage, intrinsic-to-extrinsic training paradigm: self-supervised RL on the pseudo-labeled synthetic data, followed by supervised RL on the human-annotated real data. MedSSR scales model training efficiently without relying on costly trace distillation. Extensive experiments on Qwen and Llama demonstrate that our method outperforms existing methods across ten medical benchmarks, achieving up to +5.93% gain on rare-disease tasks. Our code is available at https://github.com/tdlhl/MedSSR.

Related papers

• Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Models [43.46006663176283]
  We propose Med-Evo, a framework for medical MLLMs that utilize label-free reinforcement learning to promote model performance without requiring additional labeled data.<n>Our framework introduces two key innovations: $1)$ Feature-driven Pseudo Labeling (FPL) that identifies semantic centroids from all heterogeneous candidate responses to select pseudo labels in each rollout, and $2)$ Hard-Soft Reward (HSR) that combines exact match with token-level assessment and semantic similarity to provide hierarchical reward.
  arXiv  Detail & Related papers  (2026-03-08T03:38:24Z)
• A Reinforcement Learning Approach to Synthetic Data Generation [8.293402602656736]
  We introduce RLSyn, a novel framework that models the data generator as a policy over patient records.<n>We benchmark it against state-of-the-art generative adversarial networks (GANs) and diffusion-based methods across privacy, utility, and fidelity evaluations.
  arXiv  Detail & Related papers  (2025-12-24T19:26:37Z)
• Improving the Generation and Evaluation of Synthetic Data for Downstream Medical Causal Inference [89.5628648718851]
  Causal inference is essential for developing and evaluating medical interventions.<n>Real-world medical datasets are often difficult to access due to regulatory barriers.<n>We present STEAM: a novel method for generating Synthetic data for Treatment Effect Analysis in Medicine.
  arXiv  Detail & Related papers  (2025-10-21T16:16:00Z)
• MedGR$^2$: Breaking the Data Barrier for Medical Reasoning via Generative Reward Learning [4.579424650757833]
  Supervised Fine-Tuning (SFT) on existing datasets often leads to poor generalization on unseen modalities and tasks.<n>We introduce Generative Reward Learning for Medical Reasoning (MedGR$2$), a novel framework that creates a self-improving virtuous cycle.<n>Our experiments demonstrate that SFT with MedGR$2$-produced data already surpasses baselines trained on large-scale, human-curated datasets.
  arXiv  Detail & Related papers  (2025-08-28T08:41:32Z)
• HiRef: Leveraging Hierarchical Ontology and Network Refinement for Robust Medication Recommendation [17.45722229030237]
  We propose Hierarchical Ontology and Network Refinement for Robust Medication Recommendation (HiRef)<n>We embed entities in hyperbolic space, which naturally captures tree-like relationships and enables knowledge transfer through shared ancestors.<n>Our model achieves strong performance on EHR benchmarks (MIMIC-III and MIMIC-IV) and maintains high accuracy under simulated unseen-code settings.
  arXiv  Detail & Related papers  (2025-08-14T07:55:03Z)
• ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning [54.30630356786752]
  ReasonMed is the largest medical reasoning dataset to date, with 370k high-quality examples.<n>It is built through a multi-agent generation, verification, and refinement process.<n>Using ReasonMed, we find that integrating detailed CoT reasoning with concise answer summaries yields the most robust fine-tuning results.
  arXiv  Detail & Related papers  (2025-06-11T08:36:55Z)
• Extracting Training Data from Unconditional Diffusion Models [76.85077961718875]
  diffusion probabilistic models (DPMs) are being employed as mainstream models for generative artificial intelligence (AI) We aim to establish a theoretical understanding of memorization in DPMs with 1) a memorization metric for theoretical analysis, 2) an analysis of conditional memorization with informative and random labels, and 3) two better evaluation metrics for measuring memorization. Based on the theoretical analysis, we propose a novel data extraction method called textbfSurrogate condItional Data Extraction (SIDE) that leverages a trained on generated data as a surrogate condition to extract training data directly from unconditional diffusion models.
  arXiv  Detail & Related papers  (2024-06-18T16:20:12Z)
• Fairness-Aware Data Augmentation for Cardiac MRI using Text-Conditioned Diffusion Models [1.6581402323174208]
  We propose a method to alleviate imbalances inherent in datasets through the generation of synthetic data.<n>We adopt ControlNet based on a denoising diffusion probabilistic model to condition on text assembled from patient metadata and cardiac geometry.<n>Our experiments demonstrate the effectiveness of the proposed approach in mitigating dataset imbalances.
  arXiv  Detail & Related papers  (2024-03-28T15:41:43Z)
• Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning [5.438725298163702]
  Self-Supervised Learning (SSL) contrastive learning has shown promise in mitigating the issue of data scarcity.<n>Our research aims to explore and evaluate a wide range of audio-based augmentations and uncover combinations that enhance SSL model performance in PCG classification.
  arXiv  Detail & Related papers  (2023-12-01T11:06:00Z)
• Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data [125.7135706352493]
  Generative adversarial networks (GANs) typically require ample data for training in order to synthesize high-fidelity images. Recent studies have shown that training GANs with limited data remains formidable due to discriminator overfitting. This paper introduces a novel strategy called Adaptive Pseudo Augmentation (APA) to encourage healthy competition between the generator and the discriminator.
  arXiv  Detail & Related papers  (2021-11-12T18:13:45Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.