LesionDiffusion: Towards Text-controlled General Lesion Synthesis

• URL: http://arxiv.org/abs/2503.00741v3
• Date: Tue, 18 Mar 2025 11:31:57 GMT
• Title: LesionDiffusion: Towards Text-controlled General Lesion Synthesis
• Authors: Henrui Tian, Wenhui Lei, Linrui Dai, Hanyu Chen, Xiaofan Zhang,
• Abstract summary: We propose LesionDiffusion, a text-controllable lesion synthesis framework for 3D CT imaging.<n>Our model provides greater control over lesion attributes and supports a wider variety of lesion types.<n>We introduce a dataset of 1,505 annotated CT scans with paired lesion masks and structured reports, covering 14 lesion types across 8 organs.
• Score: 1.6029418399561406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fully-supervised lesion recognition methods in medical imaging face challenges due to the reliance on large annotated datasets, which are expensive and difficult to collect. To address this, synthetic lesion generation has become a promising approach. However, existing models struggle with scalability, fine-grained control over lesion attributes, and the generation of complex structures. We propose LesionDiffusion, a text-controllable lesion synthesis framework for 3D CT imaging that generates both lesions and corresponding masks. By utilizing a structured lesion report template, our model provides greater control over lesion attributes and supports a wider variety of lesion types. We introduce a dataset of 1,505 annotated CT scans with paired lesion masks and structured reports, covering 14 lesion types across 8 organs. LesionDiffusion consists of two components: a lesion mask synthesis network (LMNet) and a lesion inpainting network (LINet), both guided by lesion attributes and image features. Extensive experiments demonstrate that LesionDiffusion significantly improves segmentation performance, with strong generalization to unseen lesion types and organs, outperforming current state-of-the-art models. Code will be available at https://github.com/HengruiTianSJTU/LesionDiffusion.

Related papers

• Causal Disentanglement for Robust Long-tail Medical Image Generation [80.15257897500578]
  We propose a novel medical image generation framework, which generates independent pathological and structural features. We leverage a diffusion model guided by pathological findings to model pathological features, enabling the generation of diverse counterfactual images.
  arXiv  Detail & Related papers  (2025-04-20T01:54:18Z)
• Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis [55.959002385347645]
  Latent Drifting enables diffusion models to be conditioned for medical images fitted for the complex task of counterfactual image generation. We evaluate our method on three public longitudinal benchmark datasets of brain MRI and chest X-rays for counterfactual image generation.
  arXiv  Detail & Related papers  (2024-12-30T01:59:34Z)
• Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment [11.726600999078755]
  Malenia is a novel multi-scale lesion-level mask-attribute alignment framework.<n>It is specifically designed for 3D zero-shot lesion segmentation.
  arXiv  Detail & Related papers  (2024-10-21T08:01:58Z)
• Neurovascular Segmentation in sOCT with Deep Learning and Synthetic Training Data [4.5276169699857505]
  This study demonstrates a synthesis engine for neurovascular segmentation in serial-section optical coherence tomography images. Our approach comprises two phases: label synthesis and label-to-image transformation. We demonstrate the efficacy of the former by comparing it to several more realistic sets of training labels, and the latter by an ablation study of synthetic noise and artifact models.
  arXiv  Detail & Related papers  (2024-07-01T16:09:07Z)
• LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models [42.922303491557244]
  Patient data from real-world clinical practice often suffers from data scarcity and long-tail imbalances. This study addresses these challenges by generating lesion-containing image-segmentation pairs from lesion-free images. LeFusion-generated data significantly improves the performance of state-of-the-art segmentation models.
  arXiv  Detail & Related papers  (2024-03-21T01:25:39Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts [27.222844687360823]
  We adapt the latest advance in the generative model, with the added control flow using lesion-specific visual and textual prompts. It can achieve a 9% increase in the SSIM image quality measure and an over 5% increase in Dice coefficients over the prior arts.
  arXiv  Detail & Related papers  (2023-10-04T15:43:26Z)
• AMLP:Adaptive Masking Lesion Patches for Self-supervised Medical Image Segmentation [67.97926983664676]
  Self-supervised masked image modeling has shown promising results on natural images. However, directly applying such methods to medical images remains challenging. We propose a novel self-supervised medical image segmentation framework, Adaptive Masking Lesion Patches (AMLP)
  arXiv  Detail & Related papers  (2023-09-08T13:18:10Z)
• Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images [39.94162291765236]
  We present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. We employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Implicit Model (DDIM) at each step of the sampling process.
  arXiv  Detail & Related papers  (2023-08-03T21:56:50Z)
• Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images [50.76668288066681]
  We present a scale-aware super-resolution network to adaptively segment lesions of various sizes from low-resolution medical images. Our proposed network achieved consistent improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-30T14:25:55Z)
• RTNet: Relation Transformer Network for Diabetic Retinopathy Multi-lesion Segmentation [10.643730843316948]
  We find that certain lesions are closed to specific vessels and present relative patterns to each other. A self-attention transformer exploits global dependencies among lesion features, while a cross-attention transformer allows interactions between lesion and vessel features. By integrating the above blocks of dual-branches, our network segments the four kinds of lesions simultaneously.
  arXiv  Detail & Related papers  (2022-01-26T16:19:04Z)
• ElixirNet: Relation-aware Network Architecture Adaptation for Medical Lesion Detection [90.13718478362337]
  We introduce a novel ElixirNet that includes three components: 1) TruncatedRPN balances positive and negative data for false positive reduction; 2) Auto-lesion Block is automatically customized for medical images to incorporate relation-aware operations among region proposals; and 3) Relation transfer module incorporates the semantic relationship. Experiments on DeepLesion and Kits19 prove the effectiveness of ElixirNet, achieving improvement of both sensitivity and precision over FPN with fewer parameters.
  arXiv  Detail & Related papers  (2020-03-03T05:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.