CLIP-Lung: Textual Knowledge-Guided Lung Nodule Malignancy Prediction

• URL: http://arxiv.org/abs/2304.08013v1
• Date: Mon, 17 Apr 2023 06:29:14 GMT
• Title: CLIP-Lung: Textual Knowledge-Guided Lung Nodule Malignancy Prediction
• Authors: Yiming Lei, Zilong Li, Yan Shen, Junping Zhang, Hongming Shan
• Abstract summary: Lung nodule prediction has been enhanced by advanced deep-learning techniques and effective tricks. Current methods are mainly trained with cross-entropy loss using one-hot categorical labels. We propose CLIP-Lung, a textual knowledge-guided framework for lung malignancy prediction.
• Score: 34.35547775426628
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Lung nodule malignancy prediction has been enhanced by advanced deep-learning techniques and effective tricks. Nevertheless, current methods are mainly trained with cross-entropy loss using one-hot categorical labels, which results in difficulty in distinguishing those nodules with closer progression labels. Interestingly, we observe that clinical text information annotated by radiologists provides us with discriminative knowledge to identify challenging samples. Drawing on the capability of the contrastive language-image pre-training (CLIP) model to learn generalized visual representations from text annotations, in this paper, we propose CLIP-Lung, a textual knowledge-guided framework for lung nodule malignancy prediction. First, CLIP-Lung introduces both class and attribute annotations into the training of the lung nodule classifier without any additional overheads in inference. Second, we designed a channel-wise conditional prompt (CCP) module to establish consistent relationships between learnable context prompts and specific feature maps. Third, we align image features with both class and attribute features via contrastive learning, rectifying false positives and false negatives in latent space. The experimental results on the benchmark LIDC-IDRI dataset have demonstrated the superiority of CLIP-Lung, both in classification performance and interpretability of attention maps.

Related papers

• Contrastive Learning with Counterfactual Explanations for Radiology Report Generation [83.30609465252441]
  We propose a textbfCountertextbfFactual textbfExplanations-based framework (CoFE) for radiology report generation. Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking what if'' scenarios. Experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports.
  arXiv  Detail & Related papers  (2024-07-19T17:24:25Z)
• MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning [48.97640824497327]
  We propose a novel framework leveraging domain-specific medical knowledge as guiding signals to integrate language information into the visual domain through image-text contrastive learning. Our model includes global contrastive learning with our designed divergence encoder, local token-knowledge-patch alignment contrastive learning, and knowledge-guided category-level contrastive learning with expert knowledge. Notably, MLIP surpasses state-of-the-art methods even with limited annotated data, highlighting the potential of multimodal pre-training in advancing medical representation learning.
  arXiv  Detail & Related papers  (2024-02-03T05:48:50Z)
• Clinically Acceptable Segmentation of Organs at Risk in Cervical Cancer Radiation Treatment from Clinically Available Annotations [0.0]
  We present an approach to learn a deep learning model for the automatic segmentation of Organs at Risk (OARs) in cervical cancer radiation treatment. We employ simples for automatic data cleaning to minimize data inhomogeneity, label noise, and missing annotations. We develop a semi-supervised learning approach utilizing a teacher-student setup, annotation imputation, and uncertainty-guided training to learn in presence of missing annotations.
  arXiv  Detail & Related papers  (2023-02-21T13:24:40Z)
• Learning disentangled representations for explainable chest X-ray classification using Dirichlet VAEs [68.73427163074015]
  This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. The predictive capacity of multi-modal latent representations learned by DirVAE models is investigated through implementation of an auxiliary multi-label classification task.
  arXiv  Detail & Related papers  (2023-02-06T18:10:08Z)
• Non-Contrastive Learning Meets Language-Image Pre-Training [145.6671909437841]
  We study the validity of non-contrastive language-image pre-training (nCLIP) We introduce xCLIP, a multi-tasking framework combining CLIP and nCLIP, and show that nCLIP aids CLIP in enhancing feature semantics.
  arXiv  Detail & Related papers  (2022-10-17T17:57:46Z)
• PCA: Semi-supervised Segmentation with Patch Confidence Adversarial Training [52.895952593202054]
  We propose a new semi-supervised adversarial method called Patch Confidence Adrial Training (PCA) for medical image segmentation. PCA learns the pixel structure and context information in each patch to get enough gradient feedback, which aids the discriminator in convergent to an optimal state. Our method outperforms the state-of-the-art semi-supervised methods, which demonstrates its effectiveness for medical image segmentation.
  arXiv  Detail & Related papers  (2022-07-24T07:45:47Z)
• Reducing Annotation Need in Self-Explanatory Models for Lung Nodule Diagnosis [10.413504599164106]
  We propose cRedAnno, a data-/annotation-efficient self-explanatory approach for lung nodule diagnosis. cRedAnno considerably reduces the annotation need by introducing self-supervised contrastive learning. Visualisation of the learned space indicates that the correlation between the clustering of malignancy and nodule attributes coincides with clinical knowledge.
  arXiv  Detail & Related papers  (2022-06-27T20:01:41Z)
• Faithful learning with sure data for lung nodule diagnosis [34.55176532924471]
  We propose a collaborative learning framework to facilitate sure nodule classification. A loss function is designed to learn reliable features by introducing interpretability constraints regulated with nodule segmentation maps.
  arXiv  Detail & Related papers  (2022-02-25T06:33:11Z)
• DenseCLIP: Extract Free Dense Labels from CLIP [130.3830819077699]
  Contrastive Language-Image Pre-training (CLIP) has made a remarkable breakthrough in open-vocabulary zero-shot image recognition. DenseCLIP+ surpasses SOTA transductive zero-shot semantic segmentation methods by large margins. Our finding suggests that DenseCLIP can serve as a new reliable source of supervision for dense prediction tasks.
  arXiv  Detail & Related papers  (2021-12-02T09:23:01Z)
• Learning and Exploiting Interclass Visual Correlations for Medical Image Classification [30.88175218665726]
  We present the Class-Correlation Learning Network (CCL-Net) to learn interclass visual correlations from given training data. Instead of letting the network directly learn the desired correlations, we propose to learn them implicitly via distance metric learning of class-specific embeddings. An intuitive loss based on a geometrical explanation of correlation is designed for bolstering learning of the interclass correlations.
  arXiv  Detail & Related papers  (2020-07-13T13:31:38Z)
• Semi-supervised lung nodule retrieval [2.055949720959582]
  A content based image retrieval (CBIR) system provides as its output a set of images, ranked by similarity to the query image. Ground truth on similarity between dataset elements (e.g. between nodules) is not readily available, thus greatly challenging machine learning methods. The current study suggests a semi-supervised approach that involves two steps: 1) Automatic annotation of a given partially labeled dataset; 2) Learning a semantic similarity metric space based on the predicated annotations. The proposed system is demonstrated in lung nodule retrieval using the LIDC dataset, and shows that it is feasible to learn embedding from predicted ratings.
  arXiv  Detail & Related papers  (2020-05-04T19:26:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.