Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis

• URL: http://arxiv.org/abs/2507.23652v1
• Date: Thu, 31 Jul 2025 15:32:06 GMT
• Title: Adaptively Distilled ControlNet: Accelerated Training and Superior Sampling for Medical Image Synthesis
• Authors: Kunpeng Qiu, Zhiying Zhou, Yongxin Guo,
• Abstract summary: We propose textbfAdaptively Distilled ControlNet, a task-agnostic framework that accelerates training and optimization through dual-model distillation.<n>During training, a teacher model, conditioned on mask-image pairs, regularizes a mask-only student model via predicted noise alignment in parameter space.<n>During sampling, only the student model is used, enabling privacy-preserving medical image generation.
• Score: 0.5243460995467893
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image annotation is constrained by privacy concerns and labor-intensive labeling, significantly limiting the performance and generalization of segmentation models. While mask-controllable diffusion models excel in synthesis, they struggle with precise lesion-mask alignment. We propose \textbf{Adaptively Distilled ControlNet}, a task-agnostic framework that accelerates training and optimization through dual-model distillation. Specifically, during training, a teacher model, conditioned on mask-image pairs, regularizes a mask-only student model via predicted noise alignment in parameter space, further enhanced by adaptive regularization based on lesion-background ratios. During sampling, only the student model is used, enabling privacy-preserving medical image generation. Comprehensive evaluations on two distinct medical datasets demonstrate state-of-the-art performance: TransUNet improves mDice/mIoU by 2.4%/4.2% on KiTS19, while SANet achieves 2.6%/3.5% gains on Polyps, highlighting its effectiveness and superiority. Code is available at GitHub.

Related papers

• SkinDualGen: Prompt-Driven Diffusion for Simultaneous Image-Mask Generation in Skin Lesions [0.0]
  We propose a novel method that leverages the pretrained Stable Diffusion-2.0 model to generate high-quality synthetic skin lesion images.<n>A hybrid dataset combining real and synthetic data markedly enhances the performance of classification and segmentation models.
  arXiv  Detail & Related papers  (2025-07-26T15:00:37Z)
• Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation [9.795456238314825]
  We introduce Siamese-Diffusion, a novel dual-component model comprising Mask-Diffusion and Image-Diffusion.<n>During training, a Noise Consistency Loss is introduced between these components to enhance the morphological fidelity of Mask-Diffusion.
  arXiv  Detail & Related papers  (2025-05-09T14:07:27Z)
• Semi-Supervised Biomedical Image Segmentation via Diffusion Models and Teacher-Student Co-Training [7.915123555266876]
  Supervised deep learning for semantic segmentation has achieved excellent results in accurately identifying anatomical and pathological structures in medical images.<n>It often requires large annotated training datasets, which limits its scalability in clinical settings.<n>We introduce a novel semi-supervised teacher-student framework for biomedical image segmentation, inspired by the recent success of generative models.
  arXiv  Detail & Related papers  (2025-04-02T09:41:43Z)
• Frequency-Guided Masking for Enhanced Vision Self-Supervised Learning [49.275450836604726]
  We present a novel frequency-based Self-Supervised Learning (SSL) approach that significantly enhances its efficacy for pre-training.<n>We employ a two-branch framework empowered by knowledge distillation, enabling the model to take both the filtered and original images as input.
  arXiv  Detail & Related papers  (2024-09-16T15:10:07Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• Adaptive Mix for Semi-Supervised Medical Image Segmentation [22.69909762038458]
  Mix-up is a key technique for consistency regularization-based semi-supervised learning methods.<n>We propose an Adaptive Mix algorithm (AdaMix) for image mix-up in a self-paced learning manner.
  arXiv  Detail & Related papers  (2024-07-31T13:19:39Z)
• Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation for Semi-Supervised Medical Image Segmentation [13.707121013895929]
  We present a novel semi-supervised learning method, Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation. We use distinct decoders for student and teacher networks while maintain the same encoder. To learn from unlabeled data, we create pseudo-labels generated by the teacher networks and augment the training data with the pseudo-labels.
  arXiv  Detail & Related papers  (2023-08-31T09:13:34Z)
• Masked Images Are Counterfactual Samples for Robust Fine-tuning [77.82348472169335]
  Fine-tuning deep learning models can lead to a trade-off between in-distribution (ID) performance and out-of-distribution (OOD) robustness. We propose a novel fine-tuning method, which uses masked images as counterfactual samples that help improve the robustness of the fine-tuning model.
  arXiv  Detail & Related papers  (2023-03-06T11:51:28Z)
• CS$^2$: A Controllable and Simultaneous Synthesizer of Images and Annotations with Minimal Human Intervention [3.465671939864428]
  We propose a novel controllable and simultaneous synthesizer (dubbed CS$2$) to generate both realistic images and corresponding annotations at the same time. Our contributions include 1) a conditional image synthesis network that receives both style information from reference CT images and structural information from unsupervised segmentation masks, and 2) a corresponding segmentation mask network to automatically segment these synthesized images simultaneously.
  arXiv  Detail & Related papers  (2022-06-20T15:09:10Z)
• Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis [68.5287824124996]
  We present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. We apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T08:41:17Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)
• ATSO: Asynchronous Teacher-Student Optimization for Semi-Supervised Medical Image Segmentation [99.90263375737362]
  We propose ATSO, an asynchronous version of teacher-student optimization. ATSO partitions the unlabeled data into two subsets and alternately uses one subset to fine-tune the model and updates the label on the other subset. We evaluate ATSO on two popular medical image segmentation datasets and show its superior performance in various semi-supervised settings.
  arXiv  Detail & Related papers  (2020-06-24T04:05:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.