Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Models

• URL: http://arxiv.org/abs/2603.07443v1
• Date: Sun, 08 Mar 2026 03:38:24 GMT
• Title: Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Models
• Abstract summary: We propose Med-Evo, a framework for medical MLLMs that utilize label-free reinforcement learning to promote model performance without requiring additional labeled data.<n>Our framework introduces two key innovations: $1)$ Feature-driven Pseudo Labeling (FPL) that identifies semantic centroids from all heterogeneous candidate responses to select pseudo labels in each rollout, and $2)$ Hard-Soft Reward (HSR) that combines exact match with token-level assessment and semantic similarity to provide hierarchical reward.
• Score: 43.46006663176283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across diverse healthcare tasks. However, current post-training strategies, such as supervised fine-tuning and reinforcement learning, heavily depend on substantial annotated data while overlooking the potential of unlabeled test data for model enhancement. This limitation becomes particularly pronounced in medical domains, where acquiring extensive labeled medical data is difficult due to the strict data sensitivity and annotation complexity. Moreover, leveraging test data poses challenges in generating reliable supervision signals from unlabeled samples and maintaining stable self-evolution. To address these limitations, we propose Med-Evo, the first self-evolution framework for medical MLLMs that utilizes label-free reinforcement learning to promote model performance without requiring additional labeled data. Our framework introduces two key innovations: $1)$ Feature-driven Pseudo Labeling (FPL) that identifies semantic centroids from all heterogeneous candidate responses to select pseudo labels in each rollout, and $2)$ Hard-Soft Reward (HSR) that combines exact match with token-level assessment and semantic similarity to provide hierarchical reward. Experiments on three medical VQA benchmarks and two base MLLMs show clear advantages of our approach over SOTA methods, with significant improvements of 10.43\% accuracy and 4.68\% recall on the SLAKE dataset using Qwen2.5-VL, showing the effectiveness of our method.

Related papers

• Eliciting Medical Reasoning with Knowledge-enhanced Data Synthesis: A Semi-Supervised Reinforcement Learning Approach [62.0906177191353]
  Existing approaches distill chain-of-thought reasoning traces from large proprietary models via supervised fine-tuning, then conduct reinforcement learning (RL)<n>We propose MedSSR, a Medical Knowledge-enhanced data Synthesis and Semi-supervised Reinforcement learning framework.<n>Our framework first employs rare disease knowledge to synthesize distribution-controllable reasoning questions.<n>We then utilize the policy model itself to generate high-quality pseudo-labels. This enables a two-stage, intrinsic-to-extrinsic training paradigm: self-supervised RL on the pseudo-labeled data, followed by supervised RL on the human-annotated real data
  arXiv  Detail & Related papers  (2026-04-13T14:37:38Z)
• Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech [69.86604856129883]
  Dysarthric speech quality assessment (DSQA) is critical for clinical diagnostics and inclusive speech technologies.<n>We propose a three-stage framework that leverages unlabeled dysarthric speech and large-scale typical speech datasets to scale training.
  arXiv  Detail & Related papers  (2026-03-16T23:00:07Z)
• MedSeqFT: Sequential Fine-tuning Foundation Models for 3D Medical Image Segmentation [55.37355146924576]
  MedSeqFT is a sequential fine-tuning framework for medical image analysis.<n>It adapts pre-trained models to new tasks while refining their representational capacity.<n>It consistently outperforms state-of-the-art fine-tuning strategies.
  arXiv  Detail & Related papers  (2025-09-07T15:22:53Z)
• Diverse Teaching and Label Propagation for Generic Semi-Supervised Medical Image Segmentation [16.657966370261494]
  We develop a generic framework that masters all three tasks.<n>Key to solving the problem lies in how to generate reliable pseudo labels for the unlabeled data.<n>We evaluate our proposed framework on five benchmark datasets.
  arXiv  Detail & Related papers  (2025-08-12T01:33:30Z)
• Mutual Evidential Deep Learning for Medical Image Segmentation [39.930548790471896]
  We propose a mutual evidential deep learning framework that offers a potentially viable solution for pseudo-label generation in semi-supervised learning.<n>We show that MEDL achieves state-of-the-art performance in experiments on five mainstream datasets.
  arXiv  Detail & Related papers  (2025-05-18T13:42:27Z)
• High-Fidelity Pseudo-label Generation by Large Language Models for Training Robust Radiology Report Classifiers [0.2158126716116375]
  DeBERTa-RAD is a novel framework that combines the power of state-of-the-art LLM pseudo-labeling with efficient DeBERTa-based knowledge distillation for accurate and fast chest X-ray report labeling.<n> evaluated on the expert-annotated MIMIC-500 benchmark, DeBERTa-RAD achieves a state-of-the-art Macro F1 score of 0.9120.
  arXiv  Detail & Related papers  (2025-05-03T04:50:55Z)
• Semi-Supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix [59.55173022987071]
  We study the potential of semi-supervised learning for class-agnostic motion prediction. Our framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data. Our method exhibits comparable performance to weakly and some fully supervised methods.
  arXiv  Detail & Related papers  (2023-12-13T09:32:50Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging [23.08596805950814]
  We present a robust and label-efficient self-supervised FL framework for medical image analysis. Specifically, we introduce a novel distributed self-supervised pre-training paradigm into the existing FL pipeline. We show that our self-supervised FL algorithm generalizes well to out-of-distribution data and learns federated models more effectively in limited label scenarios.
  arXiv  Detail & Related papers  (2022-05-17T18:33:43Z)
• Semi-Supervised Active Learning for COVID-19 Lung Ultrasound Multi-symptom Classification [13.878896181984262]
  We propose a novel semi-supervised Two-Stream Active Learning (TSAL) method to model complicated features and reduce labeling costs. On this basis, a multi-symptom multi-label (MSML) classification network is proposed to learn discriminative features of lung symptoms. A novel lung US dataset named COVID19-LUSMS is built, currently containing 71 clinical patients with 6,836 images sampled from 678 videos.
  arXiv  Detail & Related papers  (2020-09-09T10:45:34Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.