StudyFormer : Attention-Based and Dynamic Multi View Classifier for X-ray images

• URL: http://arxiv.org/abs/2302.11840v1
• Date: Thu, 23 Feb 2023 08:03:38 GMT
• Title: StudyFormer : Attention-Based and Dynamic Multi View Classifier for X-ray images
• Authors: Lucas Wannenmacher, Michael Fitzke, Diane Wilson, Andre Dourson
• Abstract summary: We propose a novel approach for combining information from multiple views to improve the performance of X-ray image classification. Our approach is based on the use of a convolutional neural network to extract feature maps from each view, followed by an attention mechanism implemented using a Vision Transformer. The resulting model is able to perform multi-label classification on 41 labels and outperforms both single-view models and traditional multi-view classification architectures.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Chest X-ray images are commonly used in medical diagnosis, and AI models have been developed to assist with the interpretation of these images. However, many of these models rely on information from a single view of the X-ray, while multiple views may be available. In this work, we propose a novel approach for combining information from multiple views to improve the performance of X-ray image classification. Our approach is based on the use of a convolutional neural network to extract feature maps from each view, followed by an attention mechanism implemented using a Vision Transformer. The resulting model is able to perform multi-label classification on 41 labels and outperforms both single-view models and traditional multi-view classification architectures. We demonstrate the effectiveness of our approach through experiments on a dataset of 363,000 X-ray images.

Related papers

• BGM: Background Mixup for X-ray Prohibited Items Detection [75.58709178012502]
  This paper introduces a novel data augmentation approach tailored for prohibited item detection, leveraging unique characteristics inherent to X-ray imagery. Our method is motivated by observations of physical properties including: 1) X-ray Transmission Imagery: Unlike reflected light images, transmitted X-ray pixels represent composite information from multiple materials along the imaging path. We propose a simple yet effective X-ray image augmentation technique, Background Mixup (BGM), for prohibited item detection in security screening contexts.
  arXiv  Detail & Related papers  (2024-11-30T12:26:55Z)
• Dual-view X-ray Detection: Can AI Detect Prohibited Items from Dual-view X-ray Images like Humans? [78.26435264182763]
  We introduce the Large-scale Dual-view X-ray (LDXray), which consists of 353,646 instances across 12 categories. To emulate human intelligence in dual-view detection, we propose the Auxiliary-view Enhanced Network (AENet) Experiments on the LDXray dataset demonstrate that the dual-view mechanism significantly enhances detection performance.
  arXiv  Detail & Related papers  (2024-11-27T06:36:20Z)
• Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images [68.42215385041114]
  This paper introduces a novel lightweight multi-level adaptation and comparison framework to repurpose the CLIP model for medical anomaly detection. Our approach integrates multiple residual adapters into the pre-trained visual encoder, enabling a stepwise enhancement of visual features across different levels. Our experiments on medical anomaly detection benchmarks demonstrate that our method significantly surpasses current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-03-19T09:28:19Z)
• MVC: A Multi-Task Vision Transformer Network for COVID-19 Diagnosis from Chest X-ray Images [10.616065108433798]
  We propose a new method, namely Multi-task Vision Transformer (MVC) for simultaneously classifying chest X-ray images and identifying affected regions from the input data. Our method is built upon the Vision Transformer but extends its learning capability in a multi-task setting.
  arXiv  Detail & Related papers  (2023-09-30T15:52:18Z)
• XrayGPT: Chest Radiographs Summarization using Medical Vision-Language Models [60.437091462613544]
  We introduce XrayGPT, a novel conversational medical vision-language model. It can analyze and answer open-ended questions about chest radiographs. We generate 217k interactive and high-quality summaries from free-text radiology reports.
  arXiv  Detail & Related papers  (2023-06-13T17:59:59Z)
• Chest X-Rays Image Classification from beta-Variational Autoencoders Latent Features [0.0]
  We investigate and analyse the use of Deep Learning (DL) techniques to extract information from Chest X-Ray (CXR) images. We trained several beta-Variational Autoencoder (beta-VAE) models on the CheXpert dataset, one of the largest publicly available collection of labeled CXR images. Latent features have been extracted and used to train other Machine Learning models, able to classify the original images from the features extracted by the beta-VAE.
  arXiv  Detail & Related papers  (2021-09-29T23:28:09Z)
• A multi-stage GAN for multi-organ chest X-ray image generation and segmentation [2.7561479348365734]
  We present a novel multi-stage generation algorithm based on Generative Adrial Networks (GANs) Unlike other approaches, generation occurs in several stages, which simplifies the procedure and allows it to be used on very small datasets. The multistage approach achieves state-of-the-artversa and, when very few images are used to train the GANs, outperforms the corresponding single-stage approach.
  arXiv  Detail & Related papers  (2021-06-09T15:15:19Z)
• Image Embedding and Model Ensembling for Automated Chest X-Ray Interpretation [0.0]
  We present and study several machine learning approaches to develop automated Chest X-ray diagnostic models. In particular, we trained several Convolutional Neural Networks (CNN) on the CheXpert dataset. We used the trained CNNs to compute embeddings of the CXR images, in order to train two sets of tree-based classifiers from them.
  arXiv  Detail & Related papers  (2021-05-05T14:48:59Z)
• Cross-Modal Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop [63.81818077092879]
  We propose an end-to-end semi-supervised cross-modal contrastive learning framework for medical images. We first apply an image encoder to classify the chest X-rays and to generate the image features. The radiomic features are then passed through another dedicated encoder to act as the positive sample for the image features generated from the same chest X-ray.
  arXiv  Detail & Related papers  (2021-04-11T09:16:29Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.