Evaluating ChatGPT's Performance in Classifying Pneumonia from Chest X-Ray Images

• URL: http://arxiv.org/abs/2510.21839v1
• Date: Wed, 22 Oct 2025 13:31:44 GMT
• Title: Evaluating ChatGPT's Performance in Classifying Pneumonia from Chest X-Ray Images
• Abstract summary: We evaluate the ability of OpenAI's GPt-4o model to classify chest X-ray images as either NORMAL or PNEUMONIA in a zero-shot setting.<n>The results indicate that concise, feature-focused prompts achieved the highest classification accuracy of 74%.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this study, we evaluate the ability of OpenAI's gpt-4o model to classify chest X-ray images as either NORMAL or PNEUMONIA in a zero-shot setting, without any prior fine-tuning. A balanced test set of 400 images (200 from each class) was used to assess performance across four distinct prompt designs, ranging from minimal instructions to detailed, reasoning-based prompts. The results indicate that concise, feature-focused prompts achieved the highest classification accuracy of 74\%, whereas reasoning-oriented prompts resulted in lower performance. These findings highlight that while ChatGPT exhibits emerging potential for medical image interpretation, its diagnostic reliability remains limited. Continued advances in visual reasoning and domain-specific adaptation are required before such models can be safely applied in clinical practice.

Related papers

• Structured Visual Evidence Decomposition for Evidence-Grounded Multimodal Screening of Obstructive Sleep Apnea-Hypopnea Syndrome [12.95586927836034]
  EviOSAHS is an evidence-grounded multimodal reasoning framework.<n>It separates image-only anatomical evidence acquisition from final clinical adjudication.<n>It achieved 88.47% accuracy, 94.86% sensitivity, 93.74% F1-score, and a 5.14% false-negative rate.
  arXiv  Detail & Related papers  (2026-05-23T06:09:31Z)
• Towards Clinically Interpretable Ophthalmic VQA via Spatially-Grounded Lesion Evidence [51.57735743553791]
  We introduce FundusGround, a new benchmark for clinically interpretable ophthalmic VQA with spatially-grounded lesion evidence.<n>Built upon this structured lesion evidence, 72,706 questions are then generated spanning four formats.<n>Experiments demonstrate that incorporating lesion-level visual evidence consistently improves model performance and transparency.
  arXiv  Detail & Related papers  (2026-05-21T12:37:03Z)
• CRAFT: Clinical Reward-Aligned Finetuning for Medical Image Synthesis [9.138093654971831]
  We introduce the Clinical Alignment Score (CAS), a foundation-model-based proxy for clinical alignment.<n>We propose Clinical Reward-Aligned Finetuning (CRAFT), a reward-based adaptation framework.<n>CRAFT improves CAS and downstream classification performance over strong adaptation baselines.
  arXiv  Detail & Related papers  (2026-05-12T18:56:34Z)
• A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretation [48.23500302185585]
  We present CheXOne, a reasoning-enabled vision-language model for CXR interpretation.<n>CheXOne jointly generates diagnostic predictions and explicit, clinically grounded reasoning traces.<n>We show that CheXOne-drafted reports are comparable to or better than resident-written reports in 55% of cases.
  arXiv  Detail & Related papers  (2026-04-01T05:19:09Z)
• Retrieving Patient-Specific Radiomic Feature Sets for Transparent Knee MRI Assessment [42.97456036889799]
  Classical radiomic features are designed to quantify image appearance and intensity patterns.<n>Recent work on adaptive radiomics uses DL to predict feature weights over a radiomic pool, then thresholds these weights to retain the top-k features from large radiomic pool F.<n>We propose a patient-specific feature-set selection framework that predicts a single compact feature set per subject.
  arXiv  Detail & Related papers  (2026-03-02T20:12:41Z)
• A DeepSeek-Powered AI System for Automated Chest Radiograph Interpretation in Clinical Practice [83.11942224668127]
  Janus-Pro-CXR (1B) is a chest X-ray interpretation system based on DeepSeek Janus-Pro model.<n>Our system outperforms state-of-the-art X-ray report generation models in automated report generation.
  arXiv  Detail & Related papers  (2025-12-23T13:26:13Z)
• The Effect of Negation on CLIP in Medical Imaging: Limitations of Contrastive Language-Image Pretraining [2.797002704887474]
  We evaluate the Stanford AIMI CheXagent model on its ability to correctly retrieve chest X-ray images using prompts with and without negation.<n>Results from this study show improvement in handling of negation in the CLIP model with a slight decrease in accuracy of positive prompt evaluation.
  arXiv  Detail & Related papers  (2025-12-18T23:19:19Z)
• DeepGI: Explainable Deep Learning for Gastrointestinal Image Classification [0.0]
  The study confronts common endoscopic challenges such as variable lighting, fluctuating camera angles, and frequent imaging artifacts.<n>The best performing models, VGG16 and MobileNetV2, each achieved a test accuracy of 96.5%.<n>The approach includes explainable AI via Grad-CAM visualization, enabling identification of image regions most influential to model predictions.
  arXiv  Detail & Related papers  (2025-11-26T22:35:57Z)
• EchoBench: Benchmarking Sycophancy in Medical Large Vision-Language Models [82.43729208063468]
  Recent benchmarks for medical Large Vision-Language Models (LVLMs) emphasize leaderboard accuracy, overlooking reliability and safety.<n>We study sycophancy -- models' tendency to uncritically echo user-provided information.<n>We introduce EchoBench, a benchmark to systematically evaluate sycophancy in medical LVLMs.
  arXiv  Detail & Related papers  (2025-09-24T14:09:55Z)
• Evaluating Large Language Models for Multimodal Simulated Ophthalmic Decision-Making in Diabetic Retinopathy and Glaucoma Screening [37.69303106863453]
  Large language models (LLMs) can simulate clinical reasoning based on natural language prompts, but their utility in ophthalmology is largely unexplored.<n>This study evaluated GPT-4's ability to interpret structured textual descriptions of retinal fundus photographs.<n>We conducted a retrospective diagnostic validation study using 300 annotated fundus images.
  arXiv  Detail & Related papers  (2025-07-02T01:35:59Z)
• Chest Disease Detection In X-Ray Images Using Deep Learning Classification Method [0.0]
  We investigate the performance across multiple classification models to classify chest X-ray images.<n>We fine-tuned these pre-trained architectures on a labeled medical x-ray images.<n>The initial results are promising with high accuracy and strong performance in key classification metrics.
  arXiv  Detail & Related papers  (2025-05-28T17:24:33Z)
• Metrics that matter: Evaluating image quality metrics for medical image generation [48.85783422900129]
  This study comprehensively assesses commonly used no-reference image quality metrics using brain MRI data.<n>We evaluate metric sensitivity to a range of challenges, including noise, distribution shifts, and, critically, morphological alterations designed to mimic clinically relevant inaccuracies.
  arXiv  Detail & Related papers  (2025-05-12T01:57:25Z)
• Fairness Analysis of CLIP-Based Foundation Models for X-Ray Image Classification [15.98427699337596]
  We perform a comprehensive fairness analysis of CLIP-like models applied to X-ray image classification.<n>We assess their performance and fairness across diverse patient demographics and disease categories using zero-shot inference and various fine-tuning techniques.
  arXiv  Detail & Related papers  (2025-01-31T12:23:50Z)
• How Well Do Multi-modal LLMs Interpret CT Scans? An Auto-Evaluation Framework for Analyses [14.884877292068351]
  This study introduces a novel evaluation framework, named GPTRadScore'' It assesses the capabilities of multi-modal LLMs, such as GPT-4 with Vision (GPT-4V), Gemini Pro Vision, LLaVA-Med, and RadFM, in generating descriptions for prospectively-identified findings. By employing a decomposition technique based on GPT-4, GPTRadScore compares these generated descriptions with gold-standard report sentences, analyzing their accuracy in terms of body part, location, and type of finding.
  arXiv  Detail & Related papers  (2024-03-08T21:16:28Z)
• A Systematic Evaluation of GPT-4V's Multimodal Capability for Medical Image Analysis [87.25494411021066]
  GPT-4V's multimodal capability for medical image analysis is evaluated. It is found that GPT-4V excels in understanding medical images and generates high-quality radiology reports. It is found that its performance for medical visual grounding needs to be substantially improved.
  arXiv  Detail & Related papers  (2023-10-31T11:39:09Z)
• Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders [50.689585476660554]
  We propose a new fine-tuning strategy that includes positive-pair loss relaxation and random sentence sampling. Our approach consistently improves overall zero-shot pathology classification across four chest X-ray datasets and three pre-trained models.
  arXiv  Detail & Related papers  (2022-12-14T06:04:18Z)
• Preservation of High Frequency Content for Deep Learning-Based Medical Image Classification [74.84221280249876]
  An efficient analysis of large amounts of chest radiographs can aid physicians and radiologists. We propose a novel Discrete Wavelet Transform (DWT)-based method for the efficient identification and encoding of visual information.
  arXiv  Detail & Related papers  (2022-05-08T15:29:54Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.