KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification

• URL: http://arxiv.org/abs/2512.09069v1
• Date: Tue, 09 Dec 2025 19:34:30 GMT
• Title: KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
• Authors: Erfan Nourbakhsh, Nasrin Sanjari, Ali Nourbakhsh,
• Abstract summary: Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide.<n> deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands.<n>It is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency- accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD- OCT.

Related papers

• Enhanced SegNet with Integrated Grad-CAM for Interpretable Retinal Layer Segmentation in OCT Images [0.0]
  This study proposes an improved SegNet-based deep learning framework for automated and interpretable retinal layer segmentation.<n> Architectural innovations, including modified pooling strategies, enhance feature extraction from noisy OCT images.<n>Grad-CAM visualizations highlighted anatomically relevant regions, aligning segmentation with clinical biomarkers.
  arXiv  Detail & Related papers  (2025-09-09T14:31:51Z)
• Explainable Knowledge Distillation for Efficient Medical Image Classification [0.0]
  This study comprehensively explores knowledge distillation frameworks for COVID-19 and lung cancer classification using chest X-ray (CXR) images.<n>We employ high-capacity teacher models, including VGG19 and lightweight Vision Transformers, to guide the training of a compact, hardware-aware student model derived from the OFA-595 supernet.<n>We validate our models on two benchmark datasets: COVID-QU-Ex and LCS25000, covering multiple classes, including COVID-19, healthy, non-COVID pneumonia, lung, and colon cancer.
  arXiv  Detail & Related papers  (2025-08-21T05:22:47Z)
• REACT-KD: Region-Aware Cross-modal Topological Knowledge Distillation for Interpretable Medical Image Classification [2.384306241677947]
  We present REACT-KD, a framework that transfers supervision from high-fidelity multi-modal sources into a lightweight CT-based student model.<n>One branch captures structure-function relationships through dual-tracer PET/CT, while the other models dose-aware features using synthetically degraded low-dose CT.<n>The framework achieved an average AUC of 93.5% on an internal PET/CT cohort and maintained 76.6% to 81.5% AUC across varying levels of dose degradation in external CT testing.
  arXiv  Detail & Related papers  (2025-08-04T06:29:34Z)
• Comprehensive Evaluation of OCT-based Automated Segmentation of Retinal Layer, Fluid and Hyper-Reflective Foci: Impact on Clinical Assessment of Diabetic Retinopathy Severity [0.0]
  Diabetic retinopathy (DR) is a leading cause of vision loss, requiring early and accurate assessment to prevent irreversible damage.<n>This study proposes an active-learning-based deep learning pipeline for automated segmentation of retinal layers.
  arXiv  Detail & Related papers  (2025-03-03T07:23:56Z)
• EFCM: Efficient Fine-tuning on Compressed Models for deployment of large models in medical image analysis [17.876140405367764]
  This study presents an Efficient Fine-tuning on Compressed Models (EFCM) framework with two stages: unsupervised feature distillation and fine-tuning. Experiments are conducted on 11 downstream datasets related to three large medical models: RETFound for retina, MRM for chest X-ray, and BROW for histopathology.
  arXiv  Detail & Related papers  (2024-09-18T09:08:16Z)
• Co-training and Co-distillation for Quality Improvement and Compression of Language Models [88.94539115180919]
  Knowledge Distillation (KD) compresses expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models. Most smaller models fail to surpass the performance of the original larger model, resulting in sacrificing performance to improve inference speed. We propose Co-Training and Co-Distillation (CTCD), a novel framework that improves performance and inference speed together by co-training two models.
  arXiv  Detail & Related papers  (2023-11-06T03:29:00Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images [62.60956024215873]
  Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. This study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin diseases classification.
  arXiv  Detail & Related papers  (2022-03-22T06:54:29Z)
• Incremental Cross-view Mutual Distillation for Self-supervised Medical CT Synthesis [88.39466012709205]
  This paper builds a novel medical slice to increase the between-slice resolution. Considering that the ground-truth intermediate medical slices are always absent in clinical practice, we introduce the incremental cross-view mutual distillation strategy. Our method outperforms state-of-the-art algorithms by clear margins.
  arXiv  Detail & Related papers  (2021-12-20T03:38:37Z)
• Performance or Trust? Why Not Both. Deep AUC Maximization with Self-Supervised Learning for COVID-19 Chest X-ray Classifications [72.52228843498193]
  In training deep learning models, a compromise often must be made between performance and trust. In this work, we integrate a new surrogate loss with self-supervised learning for computer-aided screening of COVID-19 patients.
  arXiv  Detail & Related papers  (2021-12-14T21:16:52Z)
• Categorical Relation-Preserving Contrastive Knowledge Distillation for Medical Image Classification [75.27973258196934]
  We propose a novel Categorical Relation-preserving Contrastive Knowledge Distillation (CRCKD) algorithm, which takes the commonly used mean-teacher model as the supervisor. With this regularization, the feature distribution of the student model shows higher intra-class similarity and inter-class variance. With the contribution of the CCD and CRP, our CRCKD algorithm can distill the relational knowledge more comprehensively.
  arXiv  Detail & Related papers  (2021-07-07T13:56:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.