Integrating Vision and Location with Transformers: A Multimodal Deep Learning Framework for Medical Wound Analysis

• URL: http://arxiv.org/abs/2504.10452v1
• Date: Mon, 14 Apr 2025 17:39:18 GMT
• Title: Integrating Vision and Location with Transformers: A Multimodal Deep Learning Framework for Medical Wound Analysis
• Authors: Ramin Mousa, Hadis Taherinia, Khabiba Abdiyeva, Amir Ali Bengari, Mohammadmahdi Vahediahmar,
• Abstract summary: Deep learning (DL) has emerged as a powerful tool in wound diagnosis.<n>A body map was also created to provide location data, which can help wound specialists label wound locations more effectively.<n>The proposed model was able to achieve an accuracy of 0.8123 using image data and an accuracy of 0.8007 using a combination of image data and wound location.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Effective recognition of acute and difficult-to-heal wounds is a necessary step in wound diagnosis. An efficient classification model can help wound specialists classify wound types with less financial and time costs and also help in deciding on the optimal treatment method. Traditional machine learning models suffer from feature selection and are usually cumbersome models for accurate recognition. Recently, deep learning (DL) has emerged as a powerful tool in wound diagnosis. Although DL seems promising for wound type recognition, there is still a large scope for improving the efficiency and accuracy of the model. In this study, a DL-based multimodal classifier was developed using wound images and their corresponding locations to classify them into multiple classes, including diabetic, pressure, surgical, and venous ulcers. A body map was also created to provide location data, which can help wound specialists label wound locations more effectively. The model uses a Vision Transformer to extract hierarchical features from input images, a Discrete Wavelet Transform (DWT) layer to capture low and high frequency components, and a Transformer to extract spatial features. The number of neurons and weight vector optimization were performed using three swarm-based optimization techniques (Monster Gorilla Toner (MGTO), Improved Gray Wolf Optimization (IGWO), and Fox Optimization Algorithm). The evaluation results show that weight vector optimization using optimization algorithms can increase diagnostic accuracy and make it a very effective approach for wound detection. In the classification using the original body map, the proposed model was able to achieve an accuracy of 0.8123 using image data and an accuracy of 0.8007 using a combination of image data and wound location. Also, the accuracy of the model in combination with the optimization models varied from 0.7801 to 0.8342.

Related papers

• Efficient Brain Tumor Classification with Lightweight CNN Architecture: A Novel Approach [0.0]
  Brain tumor classification using MRI images is critical in medical diagnostics, where early and accurate detection significantly impacts patient outcomes.<n>Recent advancements in deep learning (DL) have shown promise, but many models struggle with balancing accuracy and computational efficiency.<n>We propose a novel model architecture integrating separable convolutions and squeeze and excitation (SE) blocks, designed to enhance feature extraction while maintaining computational efficiency.
  arXiv  Detail & Related papers  (2025-02-01T21:06:42Z)
• Enhancing Skin Cancer Diagnosis (SCD) Using Late Discrete Wavelet Transform (DWT) and New Swarm-Based Optimizers [0.0]
  Skin cancer stands out as one of the most life-threatening forms of cancer, with its danger amplified if not diagnosed and treated promptly.<n>Deep Learning has emerged as a powerful tool in the early detection and skin cancer diagnosis (SCD)<n>This paper proposes a novel approach to skin cancer detection, utilizing optimization techniques in conjunction with pre-trained networks and wavelet transformations.
  arXiv  Detail & Related papers  (2024-11-30T13:17:52Z)
• Brain Tumor Classification on MRI in Light of Molecular Markers [61.77272414423481]
  Co-deletion of the 1p/19q gene is associated with clinical outcomes in low-grade gliomas.<n>This study aims to utilize a specially MRI-based convolutional neural network for brain cancer detection.
  arXiv  Detail & Related papers  (2024-09-29T07:04:26Z)
• Comparative Performance Analysis of Transformer-Based Pre-Trained Models for Detecting Keratoconus Disease [0.0]
  This study compares eight pre-trained CNNs for diagnosing keratoconus, a degenerative eye disease. MobileNetV2 was the best accurate model in identifying keratoconus and normal cases with few misclassifications.
  arXiv  Detail & Related papers  (2024-08-16T20:15:24Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• An Improved Deep Convolutional Neural Network by Using Hybrid Optimization Algorithms to Detect and Classify Brain Tumor Using Augmented MRI Images [0.9990687944474739]
  In this paper, an improvement in deep convolutional learning is ensured by adopting enhanced optimization algorithms. Experimental studies are conducted to validate the performance of the suggested method on a total number of 2073 augmented MRI images. The performance comparison shows that the DCNN-G-HHO is much more successful than existing methods, especially on a scoring accuracy of 97%.
  arXiv  Detail & Related papers  (2022-06-08T14:29:06Z)
• Vision Transformers for femur fracture classification [59.99241204074268]
  The Vision Transformer (ViT) was able to correctly predict 83% of the test images. Good results were obtained in sub-fractures with the largest and richest dataset ever.
  arXiv  Detail & Related papers  (2021-08-07T10:12:42Z)
• Multiclass Burn Wound Image Classification Using Deep Convolutional Neural Networks [0.0]
  Continuous wound monitoring is important for wound specialists to allow more accurate diagnosis and optimization of management protocols. In this study, we use a deep learning-based method to classify burn wound images into two or three different categories based on the wound conditions.
  arXiv  Detail & Related papers  (2021-03-01T23:54:18Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.