Dual-Modal Lung Cancer AI: Interpretable Radiology and Microscopy with Clinical Risk Integration

• URL: http://arxiv.org/abs/2604.16104v1
• Date: Fri, 17 Apr 2026 14:37:54 GMT
• Title: Dual-Modal Lung Cancer AI: Interpretable Radiology and Microscopy with Clinical Risk Integration
• Abstract summary: This study proposes a dual-modal artificial intelligence framework that integrates CT radiology with hematoxylin and eosin histopathology for lung cancer diagnosis and subtype classification.<n>The system employs convolutional neural networks to extract radiologic and histopathologic features and incorporates clinical metadata to improve robustness.<n> Experimental results show strong performance with accuracy up to 0.87, AUROC above 0.97, and macro F1-score of 0.88.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lung cancer remains one of the leading causes of cancer-related mortality worldwide. Conventional computed tomography (CT) imaging, while essential for detection and staging, has limitations in distinguishing benign from malignant lesions and providing interpretable diagnostic insights. To address this challenge, this study proposes a dual-modal artificial intelligence framework that integrates CT radiology with hematoxylin and eosin (H&E) histopathology for lung cancer diagnosis and subtype classification. The system employs convolutional neural networks to extract radiologic and histopathologic features and incorporates clinical metadata to improve robustness. Predictions from both modalities are fused using a weighted decision-level integration mechanism to classify adenocarcinoma, squamous cell carcinoma, large cell carcinoma, small cell lung cancer, and normal tissue. Explainable AI techniques including Grad-CAM, Grad-CAM++, Integrated Gradients, Occlusion, Saliency Maps, and SmoothGrad are applied to provide visual interpretability. Experimental results show strong performance with accuracy up to 0.87, AUROC above 0.97, and macro F1-score of 0.88. Grad-CAM++ achieved the highest faithfulness and localization accuracy, demonstrating strong correspondence with expert-annotated tumor regions. These results indicate that multimodal fusion of radiology and histopathology can improve diagnostic performance while maintaining model transparency, suggesting potential for future clinical decision support systems in precision oncology.

Related papers

• Multimodal Fusion of Histopathology Images and Electronic Health Records for Early Breast Cancer Diagnosis [15.70912899013153]
  This paper presents a systematic framework that integrates patch-level histopathology features with structured clinical data.<n>We train and evaluate unimodal image models (a simple CNN baseline and ResNet-18 with transfer learning) and intermediate-fusion models (XGBoost and a multilayer perceptron)<n>Our results demonstrate that multimodal integration delivers meaningful improvements in both predictive performance and clinical transparency.
  arXiv  Detail & Related papers  (2026-04-18T19:46:34Z)
• An Explainable AI-Driven Framework for Automated Brain Tumor Segmentation Using an Attention-Enhanced U-Net [0.6087515951830486]
  We present a new segmentation technique based on a U-Net model augmented with executed attention gates to focus on the most significant regions of images.<n>To counter class imbalance, we employ manually designed loss functions like Dice Loss and Categorical Dice Loss, in conjunction with standard categorical cross-entropy.<n>Our approach achieved superior performance with accuracy of 0.9919, Dice coefficient of 0.9901, mean IoU of 0.9873, sensitivity of 0.9908, and specificity of 0.9974.
  arXiv  Detail & Related papers  (2026-03-24T15:42:49Z)
• EMeRALDS: Electronic Medical Record Driven Automated Lung Nodule Detection and Classification in Thoracic CT Images [4.533165461983661]
  Lung cancer is a leading cause of cancer-related mortality worldwide.<n>This study aims to develop a computer-aided diagnosis (CAD) system that leverages large vision-language models (VLMs)<n>The proposed approach demonstrated strong performance in zero-shot lung nodule analysis.
  arXiv  Detail & Related papers  (2025-09-15T09:11:17Z)
• MMFusion: Multi-modality Diffusion Model for Lymph Node Metastasis Diagnosis in Esophageal Cancer [13.74067035373274]
  We introduce a multi-modal heterogeneous graph-based conditional feature-guided diffusion model for lymph node metastasis diagnosis based on CT images. We propose a masked relational representation learning strategy, aiming to uncover the latent prognostic correlations and priorities of primary tumor and lymph node image representations.
  arXiv  Detail & Related papers  (2024-05-15T17:52:00Z)
• Optimizing Synthetic Correlated Diffusion Imaging for Breast Cancer Tumour Delineation [71.91773485443125]
  We show that the best AUC is achieved by the CDI$s$ - optimized modality, outperforming the best gold-standard modality by 0.0044. Notably, the optimized CDI$s$ modality also achieves AUC values over 0.02 higher than the Unoptimized CDI$s$ value.
  arXiv  Detail & Related papers  (2024-05-13T16:07:58Z)
• Survival Prediction Across Diverse Cancer Types Using Neural Networks [40.392772795903795]
  Gastric cancer and Colon adenocarcinoma represent widespread and challenging malignancies. Medical community has embraced the 5-year survival rate as a vital metric for estimating patient outcomes. This study introduces a pioneering approach to enhance survival prediction models for gastric and Colon adenocarcinoma patients.
  arXiv  Detail & Related papers  (2024-04-11T21:47:13Z)
• Exploiting Liver CT scans in Colorectal Carcinoma genomics mutation classification [0.0]
  We propose the first DeepLearning-based exploration, to our knowledge, of such classification approach from the patient medical imaging. Our method is able to identify CRC RAS mutation family from CT images with 0.73 F1 score.
  arXiv  Detail & Related papers  (2024-01-25T14:40:58Z)
• Improved Prognostic Prediction of Pancreatic Cancer Using Multi-Phase CT by Integrating Neural Distance and Texture-Aware Transformer [37.55853672333369]
  This paper proposes a novel learnable neural distance that describes the precise relationship between the tumor and vessels in CT images of different patients. The developed risk marker was the strongest predictor of overall survival among preoperative factors.
  arXiv  Detail & Related papers  (2023-08-01T12:46:02Z)
• Meta-information-aware Dual-path Transformer for Differential Diagnosis of Multi-type Pancreatic Lesions in Multi-phase CT [41.199716328468895]
  We develop a dual-path transformer to exploit the feasibility of classification and segmentation of pancreatic lesions. The proposed method consists of a CNN-based segmentation path (S-path) and a transformer-based classification path (C-path) Our results show that our method can enable accurate classification and segmentation of the full taxonomy of pancreatic lesions.
  arXiv  Detail & Related papers  (2023-03-02T03:34:28Z)
• CancerUniT: Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans [45.83431075462771]
  Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice. Most medical AI systems are built to focus on single organs with a narrow list of a few diseases. CancerUniT is a query-based Mask Transformer model with the output of multi-tumor prediction.
  arXiv  Detail & Related papers  (2023-01-28T20:09:34Z)
• Multi-Scale Hybrid Vision Transformer for Learning Gastric Histology: AI-Based Decision Support System for Gastric Cancer Treatment [50.89811515036067]
  Gastric endoscopic screening is an effective way to decide appropriate gastric cancer (GC) treatment at an early stage, reducing GC-associated mortality rate. We propose a practical AI system that enables five subclassifications of GC pathology, which can be directly matched to general GC treatment guidance.
  arXiv  Detail & Related papers  (2022-02-17T08:33:52Z)
• Hierarchical Classification of Pulmonary Lesions: A Large-Scale Radio-Pathomics Study [38.78350161086617]
  Diagnosis of pulmonary lesions from computed tomography (CT) is important but challenging for clinical decision making in lung cancer related diseases. Deep learning has achieved great success in computer aided diagnosis (CADx) area for lung cancer, whereas it suffers from label ambiguity due to the difficulty in the radiological diagnosis. Considering that invasive pathological analysis serves as the clinical golden standard of lung cancer diagnosis, in this study, we solve the label ambiguity issue via a large-scale radio-pathomics dataset. This retrospective dataset, named Pulmonary-RadPath, enables development and validation of accurate deep learning systems to predict invasive pathological labels with a non-
  arXiv  Detail & Related papers  (2020-10-08T15:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.