Meta-information-aware Dual-path Transformer for Differential Diagnosis
of Multi-type Pancreatic Lesions in Multi-phase CT
- URL: http://arxiv.org/abs/2303.00942v1
- Date: Thu, 2 Mar 2023 03:34:28 GMT
- Title: Meta-information-aware Dual-path Transformer for Differential Diagnosis
of Multi-type Pancreatic Lesions in Multi-phase CT
- Authors: Bo Zhou, Yingda Xia, Jiawen Yao, Le Lu, Jingren Zhou, Chi Liu, James
S. Duncan, Ling Zhang
- Abstract summary: We develop a dual-path transformer to exploit the feasibility of classification and segmentation of pancreatic lesions.
The proposed method consists of a CNN-based segmentation path (S-path) and a transformer-based classification path (C-path)
Our results show that our method can enable accurate classification and segmentation of the full taxonomy of pancreatic lesions.
- Score: 41.199716328468895
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Pancreatic cancer is one of the leading causes of cancer-related death.
Accurate detection, segmentation, and differential diagnosis of the full
taxonomy of pancreatic lesions, i.e., normal, seven major types of lesions, and
other lesions, is critical to aid the clinical decision-making of patient
management and treatment. However, existing works focus on segmentation and
classification for very specific lesion types (PDAC) or groups. Moreover, none
of the previous work considers using lesion prevalence-related non-imaging
patient information to assist the differential diagnosis. To this end, we
develop a meta-information-aware dual-path transformer and exploit the
feasibility of classification and segmentation of the full taxonomy of
pancreatic lesions. Specifically, the proposed method consists of a CNN-based
segmentation path (S-path) and a transformer-based classification path
(C-path). The S-path focuses on initial feature extraction by semantic
segmentation using a UNet-based network. The C-path utilizes both the extracted
features and meta-information for patient-level classification based on stacks
of dual-path transformer blocks that enhance the modeling of global contextual
information. A large-scale multi-phase CT dataset of 3,096 patients with
pathology-confirmed pancreatic lesion class labels, voxel-wise manual
annotations of lesions from radiologists, and patient meta-information, was
collected for training and evaluations. Our results show that our method can
enable accurate classification and segmentation of the full taxonomy of
pancreatic lesions, approaching the accuracy of the radiologist's report and
significantly outperforming previous baselines. Results also show that adding
the common meta-information, i.e., gender and age, can boost the model's
performance, thus demonstrating the importance of meta-information for aiding
pancreatic disease diagnosis.
Related papers
- An Attention Based Pipeline for Identifying Pre-Cancer Lesions in Head and Neck Clinical Images [1.0957311485487375]
Head and neck cancer is diagnosed in specialist centres after a surgical biopsy, but there is a potential for these to be missed leading to delayed diagnosis.
We present an attention based pipeline that identifies suspected lesions, segments, and classifies them as non-dysplastic, dysplastic and cancerous lesions.
arXiv Detail & Related papers (2024-05-03T09:02:17Z) - Multi-task Explainable Skin Lesion Classification [54.76511683427566]
We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data.
The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
arXiv Detail & Related papers (2023-10-11T05:49:47Z) - A Pathologist-Informed Workflow for Classification of Prostate Glands in
Histopathology [62.997667081978825]
Pathologists diagnose and grade prostate cancer by examining tissue from needle biopsies on glass slides.
Cancer's severity and risk of metastasis are determined by the Gleason grade, a score based on the organization and morphology of prostate cancer glands.
This paper proposes an automated workflow that follows pathologists' textitmodus operandi, isolating and classifying multi-scale patches of individual glands.
arXiv Detail & Related papers (2022-09-27T14:08:19Z) - SeATrans: Learning Segmentation-Assisted diagnosis model via Transforme [13.63128987400635]
We propose Vision-Assisted diagnosis Transformer (SeATrans) to transfer the segmentation knowledge to the disease diagnosis network.
We first propose an asymmetric multi-scale interaction strategy to correlate each single low-level diagnosis feature with multi-scale segmentation features.
To model the segmentation-diagnosis interaction, SeA-block first embeds the diagnosis feature based on the segmentation information via the encoder, and then transfers the embedding back to the diagnosis feature space by a decoder.
arXiv Detail & Related papers (2022-06-12T15:10:33Z) - Improving Classification Model Performance on Chest X-Rays through Lung
Segmentation [63.45024974079371]
We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations.
Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
arXiv Detail & Related papers (2022-02-22T15:24:06Z) - A Precision Diagnostic Framework of Renal Cell Carcinoma on Whole-Slide
Images using Deep Learning [4.823436898659051]
A deep convolutional neural network (InceptionV3) was trained on the high-quality annotated dataset of The Cancer Genome Atlas.
Our framework can help pathologists in the detection of cancer region and classification of subtypes and grades, which could be applied to any cancer type.
arXiv Detail & Related papers (2021-10-26T12:53:25Z) - BI-RADS-Net: An Explainable Multitask Learning Approach for Cancer
Diagnosis in Breast Ultrasound Images [69.41441138140895]
This paper introduces BI-RADS-Net, a novel explainable deep learning approach for cancer detection in breast ultrasound images.
The proposed approach incorporates tasks for explaining and classifying breast tumors, by learning feature representations relevant to clinical diagnosis.
Explanations of the predictions (benign or malignant) are provided in terms of morphological features that are used by clinicians for diagnosis and reporting in medical practice.
arXiv Detail & Related papers (2021-10-05T19:14:46Z) - Self-transfer learning via patches: A prostate cancer triage approach
based on bi-parametric MRI [1.3934382972253603]
Prostate cancer (PCa) is the second most common cancer diagnosed among men worldwide.
The current PCa diagnostic pathway comes at the cost of substantial overdiagnosis, leading to unnecessary treatment and further testing.
We present a patch-based pre-training strategy to distinguish between clinically significant (cS) and non-clinically significant (ncS) lesions.
arXiv Detail & Related papers (2021-07-22T17:02:38Z) - Deeply supervised UNet for semantic segmentation to assist
dermatopathological assessment of Basal Cell Carcinoma (BCC) [2.031570465477242]
We focus on detecting Basal Cell Carcinoma (BCC) through semantic segmentation using several models based on the UNet architecture.
We analyze two different encoders for the first part of the UNet network and two additional training strategies.
The best model achieves over 96%, accuracy, sensitivity, and specificity on the test set.
arXiv Detail & Related papers (2021-03-05T15:39:55Z) - G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for
Biomarker Identification and Disease Classification [49.53651166356737]
We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers.
We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
arXiv Detail & Related papers (2021-01-27T19:28:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.