Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening

• URL: http://arxiv.org/abs/2605.26283v1
• Date: Mon, 25 May 2026 19:09:35 GMT
• Title: Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening
• Abstract summary: We benchmark twelve architectures across four model families: convolutional neural networks, vision transformers, hybrid CNN-transformer backbones, and vision-language models.<n>We evaluate two tasks: binary screening for any retinal disease and multi-label classification across 28 disease classes.<n>SwinTiny and the hybrid CoAtNet0 and MaxViTTiny models achieve the strongest binary screening results and improve macro and micro F1 in the multi-label setting.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern deep learning offers powerful tools for automated retinal screening, but it remains unclear how different visual model families compare in realistic multi-disease settings and under domain shift. In this work, we benchmark twelve architectures across four model families: convolutional neural networks, vision transformers, hybrid CNN-transformer backbones, and vision-language models, using the Retinal Fundus Multi-disease Image Dataset (RFMiD). We evaluate two tasks: binary screening for any retinal disease and multi-label classification across 28 disease classes. Using standardized training, calibration, and evaluation protocols, we report AUC, F1, precision, recall, and sensitivity at a clinically relevant operating point with specificity near 80%. On RFMiD, all architectures perform well on binary screening, with AUC above 84%, but attention-based models perform best. SwinTiny and the hybrid CoAtNet0 and MaxViTTiny models achieve the strongest binary screening results and improve macro and micro F1 in the multi-label setting. Vision-language models, including CLIP ViT-B/16 and SigLIP-Base384, are competitive with CNN baselines but do not surpass the best transformer and hybrid backbones. In external validation on Messidor-2 for referable diabetic retinopathy, AUC ranges from 66.8% to 84.7%, with hybrid and transformer models again showing strong performance. These results provide a reproducible reference for model selection in multi-disease retinal screening and guide future automated screening tools for clinical deployment.

Related papers

• CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection [0.0]
  We present a unified evaluation of twelve deep learning models for binary skin cancer detection on the PAD-UFES-20 dataset.<n>Models span four families: convolutional neural networks (CNN), vision transformers (ViT), hybrid convolution transformer backbones, and vision language models (VLM)<n>Results show that well tuned CNNs already provide strong baselines, but transformer based families consistently improve discrimination.
  arXiv  Detail & Related papers  (2026-05-25T19:37:14Z)
• Multimodal Fusion of Histopathology Images and Electronic Health Records for Early Breast Cancer Diagnosis [15.70912899013153]
  This paper presents a systematic framework that integrates patch-level histopathology features with structured clinical data.<n>We train and evaluate unimodal image models (a simple CNN baseline and ResNet-18 with transfer learning) and intermediate-fusion models (XGBoost and a multilayer perceptron)<n>Our results demonstrate that multimodal integration delivers meaningful improvements in both predictive performance and clinical transparency.
  arXiv  Detail & Related papers  (2026-04-18T19:46:34Z)
• AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification [45.52399819498234]
  This work presents a systematic empirical comparison of three model paradigms for fine-grained crop disease classification.<n>We introduce AgriPath-LF16, a benchmark containing 111k images spanning 16 crops and 41 diseases.<n>CNNs achieve the highest accuracy on lab imagery but degrade under domain shift.<n> Contrastive VLMs provide a robust and parameter-efficient alternative with competitive cross-domain performance.
  arXiv  Detail & Related papers  (2026-03-08T17:28:01Z)
• A Computer Vision Hybrid Approach: CNN and Transformer Models for Accurate Alzheimer's Detection from Brain MRI Scans [0.0]
  CNN architectures consistently achieved strong performance, with ResNet50 attaining 98.83% accuracy.<n> Transformer models demonstrated competitive generalization capabilities, with ViT achieving the highest accuracy among them at 95.38%.<n>The proposed Evan_V2 hybrid model, which integrates outputs from ten CNN and Transformer architectures through feature-level fusion, achieved the best overall performance with 99.99% accuracy, 0.9989 F1-score, and ROC AUC.
  arXiv  Detail & Related papers  (2026-01-21T17:19:18Z)
• Skin Cancer Classification: Hybrid CNN-Transformer Models with KAN-Based Fusion [0.0]
  We explore Sequential and Parallel Hybrid CNN-Transformer models with Convolutional Kolmogorov-Arnold Network (CKAN)<n>Our approach integrates transfer learning and extensive data augmentation, where CNNs extract local spatial features, Transformers model global dependencies, and CKAN facilitates nonlinear feature fusion for improved representation learning.<n>Our proposed approach achieves competitive performance in skin cancer classification, demonstrating 92.81% accuracy and 92.47% F1-score on the HAM10000 dataset, 97.83% accuracy and 97.83% F1-score on the PAD-UFES dataset, and 91.17% accuracy with 91.79% F1- score on
  arXiv  Detail & Related papers  (2025-08-17T19:57:34Z)
• Artificial intelligence application in lymphoma diagnosis: from Convolutional Neural Network to Vision Transformer [34.04248949660201]
  We compare the classification performance of vision transformer to our previously designed convolutional neural network on the same dataset.<n>To the best of the authors' knowledge, this is the first direct comparison of predictive performance between a vision transformer model and a convolutional neural network model.
  arXiv  Detail & Related papers  (2025-04-05T02:33:34Z)
• Virchow: A Million-Slide Digital Pathology Foundation Model [34.38679208931425]
  We present Virchow, a foundation model for computational pathology. Virchow is a vision transformer model with 632 million parameters trained on 1.5 million hematoxylin and eosin stained whole slide images.
  arXiv  Detail & Related papers  (2023-09-14T15:09:35Z)
• Breast Ultrasound Tumor Classification Using a Hybrid Multitask CNN-Transformer Network [63.845552349914186]
  Capturing global contextual information plays a critical role in breast ultrasound (BUS) image classification. Vision Transformers have an improved capability of capturing global contextual information but may distort the local image patterns due to the tokenization operations. In this study, we proposed a hybrid multitask deep neural network called Hybrid-MT-ESTAN, designed to perform BUS tumor classification and segmentation.
  arXiv  Detail & Related papers  (2023-08-04T01:19:32Z)
• ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond [76.35955924137986]
  We propose a Vision Transformer Advanced by Exploring intrinsic IB from convolutions, i.e., ViTAE. ViTAE has several spatial pyramid reduction modules to downsample and embed the input image into tokens with rich multi-scale context. We obtain the state-of-the-art classification performance, i.e., 88.5% Top-1 classification accuracy on ImageNet validation set and the best 91.2% Top-1 accuracy on ImageNet real validation set.
  arXiv  Detail & Related papers  (2022-02-21T10:40:05Z)
• TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication [38.58585442160062]
  Multiple instance learning (MIL) is a powerful tool to solve the weakly supervised classification in whole slide image (WSI) based pathology diagnosis. We proposed a new framework, called correlated MIL, and provided a proof for convergence. We conducted various experiments for three different computational pathology problems and achieved better performance and faster convergence compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-06-02T02:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.