Learning Segmentation from Radiology Reports

• URL: http://arxiv.org/abs/2507.05582v1
• Date: Tue, 08 Jul 2025 01:37:34 GMT
• Title: Learning Segmentation from Radiology Reports
• Authors: Pedro R. A. S. Bassi, Wenxuan Li, Jieneng Chen, Zheren Zhu, Tianyu Lin, Sergio Decherchi, Andrea Cavalli, Kang Wang, Yang Yang, Alan L. Yuille, Zongwei Zhou,
• Abstract summary: Tumor segmentation in CT scans is key for diagnosis, surgery, and prognosis.<n>Yet segmentation masks are scarce because their creation requires time and expertise.<n>We propose a report-supervision loss (R-Super) that converts radiology reports into voxel-wise supervision for tumor segmentation AI.
• Score: 46.073392569125375
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Tumor segmentation in CT scans is key for diagnosis, surgery, and prognosis, yet segmentation masks are scarce because their creation requires time and expertise. Public abdominal CT datasets have from dozens to a couple thousand tumor masks, but hospitals have hundreds of thousands of tumor CTs with radiology reports. Thus, leveraging reports to improve segmentation is key for scaling. In this paper, we propose a report-supervision loss (R-Super) that converts radiology reports into voxel-wise supervision for tumor segmentation AI. We created a dataset with 6,718 CT-Report pairs (from the UCSF Hospital), and merged it with public CT-Mask datasets (from AbdomenAtlas 2.0). We used our R-Super to train with these masks and reports, and strongly improved tumor segmentation in internal and external validation--F1 Score increased by up to 16% with respect to training with masks only. By leveraging readily available radiology reports to supplement scarce segmentation masks, R-Super strongly improves AI performance both when very few training masks are available (e.g., 50), and when many masks were available (e.g., 1.7K). Project: https://github.com/MrGiovanni/R-Super

Related papers

• Unmasking Interstitial Lung Diseases: Leveraging Masked Autoencoders for Diagnosis [0.3187482513047917]
  Masked autoencoders (MAEs) have emerged as a powerful approach for pre-training on unlabelled data.<n>We train an MAE on a curated collection of over 5,000 chest computed tomography (CT) scans.<n>The pretrained MAE is then fine-tuned on a downstream classification task for diffused lung disease diagnosis.
  arXiv  Detail & Related papers  (2025-08-06T13:16:34Z)
• MAIRA-Seg: Enhancing Radiology Report Generation with Segmentation-Aware Multimodal Large Language Models [36.59952396405939]
  We introduce MAIRA-Seg, a segmentation-aware MLLM framework for radiology report generation. We train expert segmentation models to obtain mask pseudolabels for radiology-specific structures in CXRs. We employ mask-aware prompting to generate draft radiology reports.
  arXiv  Detail & Related papers  (2024-11-18T08:13:22Z)
• RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysis [56.57177181778517]
  RadGenome-Chest CT is a large-scale, region-guided 3D chest CT interpretation dataset based on CT-RATE. We leverage the latest powerful universal segmentation and large language models to extend the original datasets.
  arXiv  Detail & Related papers  (2024-04-25T17:11:37Z)
• Mask-Enhanced Segment Anything Model for Tumor Lesion Semantic Segmentation [48.107348956719775]
  We introduce Mask-Enhanced SAM (M-SAM), an innovative architecture tailored for 3D tumor lesion segmentation. We propose a novel Mask-Enhanced Adapter (MEA) within M-SAM that enriches the semantic information of medical images with positional data from coarse segmentation masks. Our M-SAM achieves high segmentation accuracy and also exhibits robust generalization.
  arXiv  Detail & Related papers  (2024-03-09T13:37:02Z)
• Advancing Radiograph Representation Learning with Masked Record Modeling [52.04899592688968]
  We formulate the self- and report-completion as two complementary objectives and present a unified framework based on masked record modeling (MRM) MRM reconstructs masked image patches and masked report tokens following a multi-task scheme to learn knowledge-enhanced semantic representations. Specifically, we find that MRM offers superior performance in label-efficient fine-tuning.
  arXiv  Detail & Related papers  (2023-01-30T18:33:32Z)
• Generative Adversarial Networks for Weakly Supervised Generation and Evaluation of Brain Tumor Segmentations on MR Images [0.0]
  This work presents a weakly supervised approach to segment anomalies in 2D magnetic resonance images. We train a generative adversarial network (GAN) that converts cancerous images to healthy variants. Non-cancerous variants can also be used to evaluate the segmentations in a weakly supervised fashion.
  arXiv  Detail & Related papers  (2022-11-10T00:04:46Z)
• A Novel Mask R-CNN Model to Segment Heterogeneous Brain Tumors through Image Subtraction [0.0]
  We propose using a method performed by radiologists called image segmentation and applying it to machine learning models to prove a better segmentation. Using Mask R-CNN, its ResNet backbone being pre-trained on the RSNA pneumonia detection challenge dataset, we can train a model on the Brats 2020 Brain Tumor dataset. We can see how well the method of image subtraction works by comparing it to models without image subtraction through DICE coefficient (F1 score), recall, and precision on the untouched test set.
  arXiv  Detail & Related papers  (2022-04-04T01:45:11Z)
• Segmentation of Lung Tumor from CT Images using Deep Supervision [0.8733639720576208]
  Lung cancer is a leading cause of death in most countries of the world. This paper approaches lung tumor segmentation by applying two-dimensional discrete wavelet transform (DWT) on the LOTUS dataset.
  arXiv  Detail & Related papers  (2021-11-17T17:50:18Z)
• Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-net neural networks: a BraTS 2020 challenge solution [56.17099252139182]
  We automate and standardize the task of brain tumor segmentation with U-net like neural networks. Two independent ensembles of models were trained, and each produced a brain tumor segmentation map. Our solution achieved a Dice of 0.79, 0.89 and 0.84, as well as Hausdorff 95% of 20.4, 6.7 and 19.5mm on the final test dataset.
  arXiv  Detail & Related papers  (2020-10-30T14:36:10Z)
• Segmentation of Lungs in Chest X-Ray Image Using Generative Adversarial Networks [0.5249805590164902]
  This paper presents the use of generative adversarial networks (GAN) to perform the task of lung segmentation on a given Chest X-ray (CXR) In our work, the generator of the GAN is trained to generate a segmented mask of a given input CXR. The objective is to generate masks for the input CXR, which are as realistic as possible compared to the ground truth masks.
  arXiv  Detail & Related papers  (2020-09-12T08:54:54Z)
• Y-Net for Chest X-Ray Preprocessing: Simultaneous Classification of Geometry and Segmentation of Annotations [70.0118756144807]
  This work introduces a general pre-processing step for chest x-ray input into machine learning algorithms. A modified Y-Net architecture based on the VGG11 encoder is used to simultaneously learn geometric orientation and segmentation of radiographs. Results were evaluated by expert clinicians, with acceptable geometry in 95.8% and annotation mask in 96.2%, compared to 27.0% and 34.9% respectively in control images.
  arXiv  Detail & Related papers  (2020-05-08T02:16:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.