FZ-VLM: A Two Stage Florence-Zephyr Vision Language Model Framework for Pulmonary Nodule Characterization and Clinical Decision Making

• URL: http://arxiv.org/abs/2608.15004v1
• Date: Sat, 15 Aug 2026 03:22:57 GMT
• Title: FZ-VLM: A Two Stage Florence-Zephyr Vision Language Model Framework for Pulmonary Nodule Characterization and Clinical Decision Making
• Abstract summary: This study presents FZ-VLM, a two-stage Florence-Zephyr Vision Language Model framework for unified structured pulmonary nodule characterization in lung CT.<n>The framework uses a fine-tuned Florence-2 model to extract radiological attributes from expert-annotated 2D axial CT slices, while a Zephyr-7B model uses these attributes to generate nodule descriptions, follow-up recommendations, and longitudinal analyses.<n>Results showed that the Stage 1 model achieved 77.18% accuracy for anatomical location, 67.96% accuracy for margin characteristics, and 79.13% accuracy for attenuation type, with a
• Score: 0.2702990676892003
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lung cancer remains one of the leading causes of cancer-related mortality worldwide, and Computed Tomography (CT) is a primary imaging tool for screening and followup assessment. After pulmonary nodule detection, radiologists manually assess anatomical location, diameter, margin characteristics, and attenuation type to support risk assessment and clinical decision-making. However, this post-detection workflow is time-consuming and can be affected by inter-observer variability. Existing Artificial Intelligence methods often focus on isolated tasks, limiting their use as a unified, clinically grounded interpretation framework. This study presents FZ-VLM, a two-stage Florence-Zephyr Vision Language Model framework for unified structured pulmonary nodule characterization in lung CT. The framework uses a fine-tuned Florence-2 model to extract radiological attributes from expert-annotated 2D axial CT slices, while a Zephyr-7B model uses these attributes to generate nodule descriptions, follow-up recommendations, and longitudinal analyses. Results showed that the Stage 1 model achieved 77.18\% accuracy for anatomical location, 67.96\% accuracy for margin characteristics, and 79.13\% accuracy for attenuation type, with a Mean Absolute Error of 2.58 mm for diameter estimation, outperforming evaluated GPT-4-based baselines as well as the human baseline. Expert radiologist evaluation of Stage 2 showed 93.9\% accuracy, 98.6\% completeness score, 76.1\% clinical relevance, and an overall score of 89.5\%. Safety analysis showed that most outputs were clinically safe, although some follow-up recommendations still required expert review. To the best of our knowledge, this study presents the first two-stage Vision-Language Model framework for structured nodule characterization and clinical decision-making.

Related papers

• Toward Vision Language Model-based Assessment of Clinical Quality and Usability of LGE-MR Images for Cardiac Ablation Planning [0.7394388288509157]
  We propose a two-stage vision language model (VLM) framework for clinically grounded image quality assessment of left atrial LGE-MRI.<n>In this work, we propose a two-stage vision language model (VLM) framework for clinically grounded image quality assessment of left atrial LGE-MRI.
  arXiv  Detail & Related papers  (2026-08-21T14:51:48Z)
• An Explainable Hybrid AI Framework for Enhanced Tuberculosis and Symptom Detection [55.35661671061754]
  Tuberculosis remains a critical global health issue, particularly in resource-limited and remote areas.<n>We propose a framework which enhances disease and symptom detection on chest X-rays by integrating two supervised heads and a self-supervised head.<n>Our model achieves an accuracy of 98.85% for distinguishing between COVID-19, tuberculosis, and normal cases, and a macro-F1 score of 90.09% for multilabel symptom detection.
  arXiv  Detail & Related papers  (2025-10-21T17:18:55Z)
• A Disease-Centric Vision-Language Foundation Model for Precision Oncology in Kidney Cancer [54.58205672910646]
  RenalCLIP is a visual-language foundation model for characterization, diagnosis and prognosis of renal mass.<n>It achieved better performance and superior generalizability across 10 core tasks spanning the full clinical workflow of kidney cancer.
  arXiv  Detail & Related papers  (2025-08-22T17:48:19Z)
• A Clinically-Grounded Two-Stage Framework for Renal CT Report Generation [4.408787333571913]
  We propose a framework for automatic renal CT report generation.<n>In Stage 1, a multi-task learning model detects structured clinical features from each 2D image.<n>In Stage 2, a vision-language model generates free-text reports conditioned on the image and the detected features.
  arXiv  Detail & Related papers  (2025-06-30T07:45:02Z)
• CT Radiomics-Based Explainable Machine Learning Model for Accurate Differentiation of Malignant and Benign Endometrial Tumors: A Two-Center Study [11.010138196822888]
  We developed a CT radiomics-based explainable machine learning model for precise diagnosing malignancy and benignity in endometrial cancer patients.<n>The diagnostic performance of the model was evaluated by using sensitivity, specificity, accuracy, precision, F1 score, AUROC, and AUPRC.<n>The Random Forest model emerged as the optimal choice for diagnosing EC, with a training AUROC of 1.00 and a testing AUROC of 0.96.
  arXiv  Detail & Related papers  (2025-06-22T17:31:06Z)
• Vision-Language Model-Based Semantic-Guided Imaging Biomarker for Lung Nodule Malignancy Prediction [0.38698178563798113]
  This research aims to integrate semantic features derived from radiologists' assessments of nodules, guiding the model to learn clinically relevant, robust, and explainable imaging features for predicting lung cancer.<n>We fine-tuned a pretrained Contrastive Language-Image Pretraining model with a parameter-efficient fine-tuning approach to align imaging and semantic text features and predict the one-year lung cancer diagnosis.
  arXiv  Detail & Related papers  (2025-04-30T06:11:34Z)
• Advanced Meta-Ensemble Machine Learning Models for Early and Accurate Sepsis Prediction to Improve Patient Outcomes [0.0]
  This paper examines the limitations of traditional sepsis screening tools like Systemic Inflammatory Response Syndrome, Modified Early Warning Score, and Quick Sequential Organ Failure Assessment. We propose using machine learning techniques - Random Forest, Extreme Gradient Boosting, and Decision Tree models - to predict sepsis onset. Our study evaluates these models individually and in a combined meta-ensemble approach using key metrics such as Accuracy, Precision, Recall, F1 score, and Area Under the Receiver Operating Characteristic Curve.
  arXiv  Detail & Related papers  (2024-07-11T00:51:32Z)
• Enhancing Diagnostic Reliability of Foundation Model with Uncertainty Estimation in OCT Images [41.002573031087856]
  We developed a foundation model with uncertainty estimation (FMUE) to detect 11 retinal conditions on optical coherence tomography ( OCT) FMUE achieved a higher F1 score of 96.76% than two state-of-the-art algorithms, RETFound and UIOS, and got further improvement with thresholding strategy to 98.44%. Our model is superior to two ophthalmologists with a higher F1 score (95.17% vs. 61.93% &71.72%)
  arXiv  Detail & Related papers  (2024-06-18T03:04:52Z)
• Penalized Deep Partially Linear Cox Models with Application to CT Scans of Lung Cancer Patients [42.09584755334577]
  Lung cancer is a leading cause of cancer mortality globally, highlighting the importance of understanding its mortality risks to design effective therapies. The National Lung Screening Trial (NLST) employed computed tomography texture analysis to quantify the mortality risks of lung cancer patients. We propose a novel Penalized Deep Partially Linear Cox Model (Penalized DPLC), which incorporates the SCAD penalty to select important texture features and employs a deep neural network to estimate the nonparametric component of the model.
  arXiv  Detail & Related papers  (2023-03-09T15:38:16Z)
• Validated respiratory drug deposition predictions from 2D and 3D medical images with statistical shape models and convolutional neural networks [47.187609203210705]
  We aim to develop and validate an automated computational framework for patient-specific deposition modelling. An image processing approach is proposed that could produce 3D patient respiratory geometries from 2D chest X-rays and 3D CT images.
  arXiv  Detail & Related papers  (2023-03-02T07:47:07Z)
• Severity Assessment of Coronavirus Disease 2019 (COVID-19) Using Quantitative Features from Chest CT Images [54.919022945740515]
  The aim of this study is to realize automatic severity assessment (non-severe or severe) of COVID-19 based on chest CT images. A random forest (RF) model is trained to assess the severity (non-severe or severe) based on quantitative features. Several quantitative features, which have the potential to reflect the severity of COVID-19, were revealed.
  arXiv  Detail & Related papers  (2020-03-26T15:49:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.