Vision-Language Model-Based Semantic-Guided Imaging Biomarker for Lung Nodule Malignancy Prediction

• URL: http://arxiv.org/abs/2504.21344v3
• Date: Sat, 01 Nov 2025 19:04:13 GMT
• Title: Vision-Language Model-Based Semantic-Guided Imaging Biomarker for Lung Nodule Malignancy Prediction
• Authors: Luoting Zhuang, Seyed Mohammad Hossein Tabatabaei, Ramin Salehi-Rad, Linh M. Tran, Denise R. Aberle, Ashley E. Prosper, William Hsu,
• Abstract summary: This research aims to integrate semantic features derived from radiologists' assessments of nodules, guiding the model to learn clinically relevant, robust, and explainable imaging features for predicting lung cancer.<n>We fine-tuned a pretrained Contrastive Language-Image Pretraining model with a parameter-efficient fine-tuning approach to align imaging and semantic text features and predict the one-year lung cancer diagnosis.
• Score: 0.38698178563798113
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning models have utilized semantic features, deep features, or both to assess lung nodule malignancy. However, their reliance on manual annotation during inference, limited interpretability, and sensitivity to imaging variations hinder their application in real-world clinical settings. Thus, this research aims to integrate semantic features derived from radiologists' assessments of nodules, guiding the model to learn clinically relevant, robust, and explainable imaging features for predicting lung cancer. We obtained 938 low-dose CT scans from the National Lung Screening Trial (NLST) with 1,261 nodules and semantic features. Additionally, the Lung Image Database Consortium dataset contains 1,018 CT scans, with 2,625 lesions annotated for nodule characteristics. Three external datasets were obtained from UCLA Health, the LUNGx Challenge, and the Duke Lung Cancer Screening. We fine-tuned a pretrained Contrastive Language-Image Pretraining (CLIP) model with a parameter-efficient fine-tuning approach to align imaging and semantic text features and predict the one-year lung cancer diagnosis. Our model outperformed state-of-the-art (SOTA) models in the NLST test set with an AUROC of 0.901 and AUPRC of 0.776. It also showed robust results in external datasets. Using CLIP, we also obtained predictions on semantic features through zero-shot inference, such as nodule margin (AUROC: 0.807), nodule consistency (0.812), and pleural attachment (0.840). Our approach surpasses the SOTA models in predicting lung cancer across datasets collected from diverse clinical settings, providing explainable outputs, aiding clinicians in comprehending the underlying meaning of model predictions. This approach also prevents the model from learning shortcuts and generalizes across clinical settings. The code is available at https://github.com/luotingzhuang/CLIP_nodule.

Related papers

• LungEvaty: A Scalable, Open-Source Transformer-based Deep Learning Model for Lung Cancer Risk Prediction in LDCT Screening [37.29507297342265]
  LungEvaty is a transformer-based framework for predicting 1-6 year lung cancer risk from a single LDCT scan.<n>It learns directly from large-scale screening data to capture comprehensive anatomical and pathological cues relevant for malignancy risk.<n>LungEvaty was trained on more than 90,000 CT scans, including over 28,000 for fine-tuning and 6,000 for evaluation.
  arXiv  Detail & Related papers  (2025-11-25T09:38:10Z)
• LMLCC-Net: A Semi-Supervised Deep Learning Model for Lung Nodule Malignancy Prediction from CT Scans using a Novel Hounsfield Unit-Based Intensity Filtering [1.2974519529978974]
  We propose LMLCC-Net, a novel deep learning framework for classifying nodules from CT scan images using a 3D CNN.<n>Our proposed method achieves a classification accuracy (ACC) of 91.96%, a sensitivity (SEN) of 92.04%, and an area under the curve (AUC) of 91.87%.
  arXiv  Detail & Related papers  (2025-05-09T18:25:59Z)
• Patient-specific radiomic feature selection with reconstructed healthy persona of knee MR images [39.11295870085984]
  Classical radiomic features have been designed to describe image appearance and intensity patterns.<n>Lower dimensional parametric models that use such radiomic features offer enhanced interpretability but lower comparative performance in clinical tasks.<n>We propose an approach where a standard logistic regression model performance is substantially improved by learning to select radiomic features for individual patients.
  arXiv  Detail & Related papers  (2025-03-17T12:55:43Z)
• Fast-staged CNN Model for Accurate pulmonary diseases and Lung cancer detection [0.0]
  This research evaluates a deep learning model designed to detect lung cancer, specifically pulmonary nodules, along with eight other lung pathologies, using chest radiographs.<n>A two-stage classification system, utilizing ensemble methods and transfer learning, is employed to first triage images into Normal or Abnormal.<n>The model achieves notable results in classification, with a top-performing accuracy of 77%, a sensitivity of 0.713, a specificity of 0.776 during external validation, and an AUC score of 0.888.
  arXiv  Detail & Related papers  (2024-12-16T11:47:07Z)
• Multilabel Classification for Lung Disease Detection: Integrating Deep Learning and Natural Language Processing [0.0]
  We propose a novel transfer learning model for multi-label lung disease classification.<n>The proposed model achieved an F1 score of 0.69 and an AUROC of 0.86, demonstrating its potential for clinical applications.
  arXiv  Detail & Related papers  (2024-12-16T05:14:08Z)
• Feature-interactive Siamese graph encoder-based image analysis to predict STAS from histopathology images in lung cancer [7.773244667755116]
  Histopathology is the gold standard for STAS detection, yet traditional methods are subjective, time-consuming, and prone to misdiagnosis. We present VERN, an image analysis model utilizing a feature-interactive Siamese graph encoder to predict STAS from lung cancer images.
  arXiv  Detail & Related papers  (2024-11-22T14:21:52Z)
• Detection-Guided Deep Learning-Based Model with Spatial Regularization for Lung Nodule Segmentation [2.4044422838107438]
  Lung cancer ranks as one of the leading causes of cancer diagnosis and is the foremost cause of cancer-related mortality worldwide. The segmentation of lung nodules plays a critical role in aiding physicians in distinguishing between malignant and benign lesions. We introduce a novel model for segmenting lung nodules in computed tomography (CT) images, leveraging a deep learning framework that integrates segmentation and classification processes.
  arXiv  Detail & Related papers  (2024-10-26T11:58:12Z)
• Lung-CADex: Fully automatic Zero-Shot Detection and Classification of Lung Nodules in Thoracic CT Images [45.29301790646322]
  Computer-aided diagnosis can help with early lung nodul detection and facilitate subsequent nodule characterization. We propose CADe, for segmenting lung nodules in a zero-shot manner using a variant of the Segment Anything Model called MedSAM. We also propose, CADx, a method for the nodule characterization as benign/malignant by making a gallery of radiomic features and aligning image-feature pairs through contrastive learning.
  arXiv  Detail & Related papers  (2024-07-02T19:30:25Z)
• Variational Autoencoders for Feature Exploration and Malignancy Prediction of Lung Lesions [0.0]
  Lung cancer is responsible for 21% of cancer deaths in the UK. Recent studies have demonstrated the capability of AI methods for accurate and early diagnosis of lung cancer from routine scans. This study investigates the application Variational Autoencoders (VAEs), a type of generative AI model, to lung cancer lesions.
  arXiv  Detail & Related papers  (2023-11-27T11:12:33Z)
• A Two-Stage Generative Model with CycleGAN and Joint Diffusion for MRI-based Brain Tumor Detection [41.454028276986946]
  We propose a novel framework Two-Stage Generative Model (TSGM) to improve brain tumor detection and segmentation. CycleGAN is trained on unpaired data to generate abnormal images from healthy images as data prior. VE-JP is implemented to reconstruct healthy images using synthetic paired abnormal images as a guide.
  arXiv  Detail & Related papers  (2023-11-06T12:58:26Z)
• Classification of lung cancer subtypes on CT images with synthetic pathological priors [41.75054301525535]
  Cross-scale associations exist in the image patterns between the same case's CT images and its pathological images. We propose self-generating hybrid feature network (SGHF-Net) for accurately classifying lung cancer subtypes on CT images.
  arXiv  Detail & Related papers  (2023-08-09T02:04:05Z)
• Penalized Deep Partially Linear Cox Models with Application to CT Scans of Lung Cancer Patients [42.09584755334577]
  Lung cancer is a leading cause of cancer mortality globally, highlighting the importance of understanding its mortality risks to design effective therapies. The National Lung Screening Trial (NLST) employed computed tomography texture analysis to quantify the mortality risks of lung cancer patients. We propose a novel Penalized Deep Partially Linear Cox Model (Penalized DPLC), which incorporates the SCAD penalty to select important texture features and employs a deep neural network to estimate the nonparametric component of the model.
  arXiv  Detail & Related papers  (2023-03-09T15:38:16Z)
• Deep-Learning Tool for Early Identifying Non-Traumatic Intracranial Hemorrhage Etiology based on CT Scan [40.51754649947294]
  The deep learning model was developed with 1868 eligible NCCT scans with non-traumatic ICH collected between January 2011 and April 2018. The model's diagnostic performance was compared with clinicians's performance. The clinicians achieve significant improvements in the sensitivity, specificity, and accuracy of diagnoses of certain hemorrhage etiologies with proposed system augmentation.
  arXiv  Detail & Related papers  (2023-02-02T08:45:17Z)
• 3D-Morphomics, Morphological Features on CT scans for lung nodule malignancy diagnosis [8.728543774561405]
  The study develops a predictive model of the pathological states based on morphological features (3D-morphomics) on Computed Tomography (CT) volumes. An XGBoost supervised classifier is then trained and tested on the 3D-morphomics to predict the pathological states. Using 3D-morphomics only, the classification model of lung nodules into malignant vs. benign achieves 0.964 of AUC.
  arXiv  Detail & Related papers  (2022-07-27T23:50:47Z)
• Integrative Analysis for COVID-19 Patient Outcome Prediction [53.11258640541513]
  We combine radiomics of lung opacities and non-imaging features from demographic data, vital signs, and laboratory findings to predict need for intensive care unit admission. Our methods may also be applied to other lung diseases including but not limited to community acquired pneumonia.
  arXiv  Detail & Related papers  (2020-07-20T19:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.