ViT-2SPN: Vision Transformer-based Dual-Stream Self-Supervised Pretraining Networks for Retinal OCT Classification

• URL: http://arxiv.org/abs/2501.17260v1
• Date: Tue, 28 Jan 2025 19:41:38 GMT
• Title: ViT-2SPN: Vision Transformer-based Dual-Stream Self-Supervised Pretraining Networks for Retinal OCT Classification
• Abstract summary: Vision Transformer-based Dual-Stream Self-Supervised Pretraining Network (ViT-2SPN) is a novel framework designed to enhance feature extraction and improve diagnostic accuracy.<n>ViT-2SPN employs a three-stage workflow: Supervised Pretraining, Self-Supervised Pretraining, and Supervised Fine-Tuning.<n>ViT-2SPN achieves a mean AUC of 0.93, accuracy of 0.77, precision of 0.81, recall of 0.75, and an F1 score of 0.76, outperforming existing SSP-based methods.
• Score: 0.10241134756773226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Optical Coherence Tomography (OCT) is a non-invasive imaging modality essential for diagnosing various eye diseases. Despite its clinical significance, developing OCT-based diagnostic tools faces challenges, such as limited public datasets, sparse annotations, and privacy concerns. Although deep learning has made progress in automating OCT analysis, these challenges remain unresolved. To address these limitations, we introduce the Vision Transformer-based Dual-Stream Self-Supervised Pretraining Network (ViT-2SPN), a novel framework designed to enhance feature extraction and improve diagnostic accuracy. ViT-2SPN employs a three-stage workflow: Supervised Pretraining, Self-Supervised Pretraining (SSP), and Supervised Fine-Tuning. The pretraining phase leverages the OCTMNIST dataset (97,477 unlabeled images across four disease classes) with data augmentation to create dual-augmented views. A Vision Transformer (ViT-Base) backbone extracts features, while a negative cosine similarity loss aligns feature representations. Pretraining is conducted over 50 epochs with a learning rate of 0.0001 and momentum of 0.999. Fine-tuning is performed on a stratified 5.129% subset of OCTMNIST using 10-fold cross-validation. ViT-2SPN achieves a mean AUC of 0.93, accuracy of 0.77, precision of 0.81, recall of 0.75, and an F1 score of 0.76, outperforming existing SSP-based methods.

Related papers

• Post-Operative Glioma Segmentation via Loss Stabilization, Normalization and Subspace Attention [46.03321798937855]
  We show that the standard Generalized Dice Loss (GDL) is unstable under domain shift.<n>We propose a Subspace-Aware Class Attention (SACA) module that re-calibrates the bottleneck features and raises Enhancing Tumor (ET) sensitivity by 8%.
  arXiv  Detail & Related papers  (2026-07-23T09:59:30Z)
• Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography [75.7789001619107]
  Vision-language pre-training holds great promise for general-purpose medical AI.<n>We propose a tailored framework featuring three key components.<n>A diagnosis-aware prompt strategy employs real clinical phrases and aggregated disease prototypes to bridge the pre-training-inference gap.
  arXiv  Detail & Related papers  (2026-06-24T08:24:45Z)
• Self-Supervised Vision Transformers for CBCT-Based Detection of Temporomandibular Joint Osteoarthritis [0.0]
  Temporomandibular joint osteoarthritis (TMJ OA) is a prevalent degenerative condition whose osseous changes are often subtle on cone-beam CT (CBCT)<n>We study how well the DINO family of self-supervised vision transformers transfers to CBCT.<n>We propose a simple slice-based pipeline using Vision Transformer (ViT) backbones.
  arXiv  Detail & Related papers  (2026-06-06T22:42:17Z)
• Artificial intelligence application in lymphoma diagnosis with Vision Transformer using weakly supervised training [3.881131438027152]
  Vision transformers (ViT) have been shown to allow for more flexible feature detection and can outperform convolutional neural network (CNN) when pre-trained on sufficient data.<n>We deploy ViTs for morphological classification of anaplastic large cell lymphoma (ALCL) versus classic Hodgkin lymphoma (cHL)<n>Our ViT model, trained on a larger dataset of 100,000 image patches, yields evaluation metrics with significant accuracy, F1 score, and area under the curve (AUC) at 91.85%, 0.92, and 0.98, respectively.
  arXiv  Detail & Related papers  (2026-04-15T12:33:17Z)
• Distilling Photon-Counting CT into Routine Chest CT through Clinically Validated Degradation Modeling [51.330452704464925]
  We train a latent diffusion model on 1,046 PCCTs and 405,379 EICTs from 145 hospitals.<n>We construct a large-scale dataset of over 17,316 publicly available EICTs enhanced to PCCT-like quality.<n>Our results suggest emerging imaging advances can be systematically distilled into routine EICT using limited high-quality scans as reference.
  arXiv  Detail & Related papers  (2026-04-08T17:47:35Z)
• Virtual-Eyes: Quantitative Validation of a Lung CT Quality-Control Pipeline for Foundation-Model Cancer Risk Prediction [0.0]
  Preprocessing is rarely quantified in deep-learning pipelines for low-dose lung cancer screening.<n>We develop VirtualEyes, a 16-bit CT quality-control pipeline, and measure its impact on generalist foundation models versus specialist models.
  arXiv  Detail & Related papers  (2025-12-30T15:34:18Z)
• Patch-Level Glioblastoma Subregion Classification with a Contrastive Learning-Based Encoder [8.220201308071614]
  We developed a method that fine-tunes a pre-trained Vision Transformer (ViT) encoder with a dedicated classification head on the official training dataset.<n>Our model's performance on the online validation set, evaluated via the Synapse platform, yielded a Matthews Correlation Coefficient (MCC) of 0.7064 and an F1-score of 0.7676.<n>On the final test set, the model achieved an MCC of 0.6509 and an F1-score of 0.5330, which secured our team second place in the BraTS-Pathology 2025 Challenge.
  arXiv  Detail & Related papers  (2025-11-25T11:49:18Z)
• Validating Vision Transformers for Otoscopy: Performance and Data-Leakage Effects [42.465094107111646]
  This study evaluates the efficacy of vision transformer models, specifically Swin transformers, in enhancing the diagnostic accuracy of ear diseases.<n>The research utilised a real-world dataset from the Department of Otolaryngology at the Clinical Hospital of the Universidad de Chile.
  arXiv  Detail & Related papers  (2025-11-06T23:20:37Z)
• SegFormer Fine-Tuning with Dropout: Advancing Hair Artifact Removal in Skin Lesion Analysis [0.0]
  Hair artifacts in dermoscopic images present significant challenges for accurate skin lesion analysis.<n>This work introduces a fine-tuned SegFormer model augmented with dropout regularization to achieve precise hair mask segmentation.
  arXiv  Detail & Related papers  (2025-09-02T10:06:26Z)
• MedNet-PVS: A MedNeXt-Based Deep Learning Model for Automated Segmentation of Perivascular Spaces [7.639673588124668]
  Enlarged perivascular spaces (PVS) are recognized as biomarkers of cerebral small vessel disease, Alzheimer's disease, stroke, and aging-related neurodegeneration.<n>We adapted MedNeXt-L-k5, a Transformer-inspired 3D encoder-decoder convolutional network, for automated PVS segmentation.
  arXiv  Detail & Related papers  (2025-08-27T20:24:12Z)
• A Novel Attention-Augmented Wavelet YOLO System for Real-time Brain Vessel Segmentation on Transcranial Color-coded Doppler [49.03919553747297]
  We propose an AI-powered, real-time CoW auto-segmentation system capable of efficiently capturing cerebral arteries.<n>No prior studies have explored AI-driven cerebrovascular segmentation using Transcranial Color-coded Doppler (TCCD)<n>The proposed AAW-YOLO demonstrated strong performance in segmenting both ipsilateral and contralateral CoW vessels.
  arXiv  Detail & Related papers  (2025-08-19T14:41:22Z)
• FUTransUNet-GradCAM: A Hybrid Transformer-U-Net with Self-Attention and Explainable Visualizations for Foot Ulcer Segmentation [0.0]
  Automated segmentation of diabetic foot ulcers (DFUs) plays a critical role in clinical diagnosis, therapeutic planning, and longitudinal wound monitoring.<n>Traditional convolutional neural networks (CNNs) provide strong localization capabilities but struggle to model long-range spatial dependencies.<n>We propose FUTransUNet, a hybrid architecture that integrates the global attention mechanism of Vision Transformers (ViTs) into the U-Net framework.
  arXiv  Detail & Related papers  (2025-08-04T11:05:14Z)
• Enhancing Transfer Learning for Medical Image Classification with SMOTE: A Comparative Study [0.0]
  This paper explores and enhances the application of Transfer Learning (TL) for multilabel image classification in medical imaging. Our results show that TL models excel in brain tumor classification, achieving near-optimal metrics. We integrate the Synthetic Minority Oversampling computation Technique (SMOTE) with TL and traditional machine learning(ML) methods, which improves accuracy by 1.97%, recall (sensitivity) by 5.43%, and specificity by 0.72%.
  arXiv  Detail & Related papers  (2024-12-28T18:15:07Z)
• Multi-Class Abnormality Classification Task in Video Capsule Endoscopy [3.656114607436271]
  This work addressed the challenge of multiclass anomaly classification in video capsule Endoscopy (VCE)<n>The purpose is to correctly classify diverse gastrointestinal disorders, which is critical for increasing diagnostic efficiency in clinical settings.<n>Our team capsule commandos achieved 7th place ranking with a test set[7] performance of Mean AUC: 0.7314 and balanced accuracy: 0.3235.
  arXiv  Detail & Related papers  (2024-10-25T21:22:52Z)
• Integrating Deep Learning with Fundus and Optical Coherence Tomography for Cardiovascular Disease Prediction [47.7045293755736]
  Early identification of patients at risk of cardiovascular diseases (CVD) is crucial for effective preventive care, reducing healthcare burden, and improving patients' quality of life. This study demonstrates the potential of retinal optical coherence tomography ( OCT) imaging combined with fundus photographs for identifying future adverse cardiac events. We propose a novel binary classification network based on a Multi-channel Variational Autoencoder (MCVAE), which learns a latent embedding of patients' fundus and OCT images to classify individuals into two groups: those likely to develop CVD in the future and those who are not.
  arXiv  Detail & Related papers  (2024-10-18T12:37:51Z)
• Phikon-v2, A large and public feature extractor for biomarker prediction [42.52549987351643]
  We train a vision transformer using DINOv2 and publicly release one iteration of this model for further experimentation, coined Phikon-v2. While trained on publicly available histology slides, Phikon-v2 surpasses our previously released model (Phikon) and performs on par with other histopathology foundation models (FM) trained on proprietary data.
  arXiv  Detail & Related papers  (2024-09-13T20:12:29Z)
• Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastoma [4.578027879885667]
  This research aims to improve glioblastoma survival prediction by integrating MR images, clinical and molecular-pathologic data in a transformer-based deep learning model. The model employs self-supervised learning techniques to effectively encode the high-dimensional MRI input for integration with non-imaging data using cross-attention.
  arXiv  Detail & Related papers  (2024-05-21T17:44:48Z)
• Virtual imaging trials improved the transparency and reliability of AI systems in COVID-19 imaging [1.6040478776985583]
  This study focuses on using convolutional neural networks (CNNs) for COVID-19 diagnosis using computed tomography (CT) and chest radiography (CXR) We developed and tested multiple AI models, 3D ResNet-like and 2D EfficientNetv2 architectures, across diverse datasets. Models trained on the most diverse datasets showed the highest external testing performance, with AUC values ranging from 0.73 to 0.76 for CT and 0.70 to 0.73 for CXR.
  arXiv  Detail & Related papers  (2023-08-17T19:12:32Z)
• Bi-ViT: Pushing the Limit of Vision Transformer Quantization [38.24456467950003]
  Vision transformers (ViTs) quantization offers a promising prospect to facilitate deploying large pre-trained networks on resource-limited devices. We introduce a learnable scaling factor to reactivate the vanished gradients and illustrate its effectiveness through theoretical and experimental analyses. We then propose a ranking-aware distillation method to rectify the disordered ranking in a teacher-student framework.
  arXiv  Detail & Related papers  (2023-05-21T05:24:43Z)
• Cross-Shaped Windows Transformer with Self-supervised Pretraining for Clinically Significant Prostate Cancer Detection in Bi-parametric MRI [6.930082824262643]
  We introduce a novel end-to-end Cross-Shaped windows (CSwin) transformer UNet model, CSwin UNet, to detect clinically significant prostate cancer (csPCa) in prostate bi-parametric MR imaging (bpMRI) Using a large prostate bpMRI dataset with 1500 patients, we first pretrain CSwin transformer using multi-task self-supervised learning to improve data-efficiency and network generalizability. Five-fold cross validation shows that self-supervised CSwin UNet achieves 0.888 AUC and 0.545 Average Precision (AP), significantly outperforming four comparable models (Swin U
  arXiv  Detail & Related papers  (2023-04-30T04:40:32Z)
• Tissue Classification During Needle Insertion Using Self-Supervised Contrastive Learning and Optical Coherence Tomography [53.38589633687604]
  We propose a deep neural network that classifies the tissues from the phase and intensity data of complex OCT signals acquired at the needle tip. We show that with 10% of the training set, our proposed pretraining strategy helps the model achieve an F1 score of 0.84 whereas the model achieves an F1 score of 0.60 without it.
  arXiv  Detail & Related papers  (2023-04-26T14:11:04Z)
• Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosis [48.64462717254158]
  We developed a self-supervised contrastive learning approach, EchoCLR, to catered to echocardiogram videos. When fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improved classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS) EchoCLR is unique in its ability to learn representations of medical videos and demonstrates that SSL can enable label-efficient disease classification from small, labeled datasets.
  arXiv  Detail & Related papers  (2022-07-23T19:17:26Z)
• Vision Transformers for femur fracture classification [59.99241204074268]
  The Vision Transformer (ViT) was able to correctly predict 83% of the test images. Good results were obtained in sub-fractures with the largest and richest dataset ever.
  arXiv  Detail & Related papers  (2021-08-07T10:12:42Z)
• Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images [49.1861463923357]
  We propose an uncertainty-guided dual-consistency learning network (UDC-Net) for semi-supervised COVID-19 lesion segmentation from CT images. Our proposed UDC-Net improves the fully supervised method by 6.3% in Dice and outperforms other competitive semi-supervised approaches by significant margins.
  arXiv  Detail & Related papers  (2021-04-07T16:23:35Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)
• CovidDeep: SARS-CoV-2/COVID-19 Test Based on Wearable Medical Sensors and Efficient Neural Networks [51.589769497681175]
  The novel coronavirus (SARS-CoV-2) has led to a pandemic. The current testing regime based on Reverse Transcription-Polymerase Chain Reaction for SARS-CoV-2 has been unable to keep up with testing demands. We propose a framework called CovidDeep that combines efficient DNNs with commercially available WMSs for pervasive testing of the virus.
  arXiv  Detail & Related papers  (2020-07-20T21:47:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.