Vision-Language Models for Acute Tuberculosis Diagnosis: A Multimodal Approach Combining Imaging and Clinical Data

• URL: http://arxiv.org/abs/2503.14538v3
• Date: Tue, 01 Apr 2025 06:41:57 GMT
• Title: Vision-Language Models for Acute Tuberculosis Diagnosis: A Multimodal Approach Combining Imaging and Clinical Data
• Authors: Ananya Ganapthy, Praveen Shastry, Naveen Kumarasami, Anandakumar D, Keerthana R, Mounigasri M, Varshinipriya M, Kishore Prasath Venkatesh, Bargava Subramanian, Kalyan Sivasailam,
• Abstract summary: This study introduces a Vision-Language Model (VLM) leveraging SIGLIP and Gemma-3b architectures for automated acute tuberculosis (TB) screening.<n>The VLM combines visual data from chest X-rays with clinical context to generate detailed, context-aware diagnostic reports.<n>Key acute TB pathologies, including consolidation, cavities, and nodules, were detected with high precision and recall.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background: This study introduces a Vision-Language Model (VLM) leveraging SIGLIP and Gemma-3b architectures for automated acute tuberculosis (TB) screening. By integrating chest X-ray images and clinical notes, the model aims to enhance diagnostic accuracy and efficiency, particularly in resource-limited settings. Methods: The VLM combines visual data from chest X-rays with clinical context to generate detailed, context-aware diagnostic reports. The architecture employs SIGLIP for visual encoding and Gemma-3b for decoding, ensuring effective representation of acute TB-specific pathologies and clinical insights. Results: Key acute TB pathologies, including consolidation, cavities, and nodules, were detected with high precision (97percent) and recall (96percent). The model demonstrated strong spatial localization capabilities and robustness in distinguishing TB-positive cases, making it a reliable tool for acute TB diagnosis. Conclusion: The multimodal capability of the VLM reduces reliance on radiologists, providing a scalable solution for acute TB screening. Future work will focus on improving the detection of subtle pathologies and addressing dataset biases to enhance its generalizability and application in diverse global healthcare settings.

Related papers

• Investigating Zero-Shot Diagnostic Pathology in Vision-Language Models with Efficient Prompt Design [7.509731425152396]
  We present a systematic investigation and analysis of three state of the art vision-language models (VLMs) for histopathology. We develop a comprehensive prompt engineering framework that systematically varies domain specificity, anatomical precision, instructional framing, and output constraints. Our findings demonstrate that prompt engineering significantly impacts model performance, with the CONCH model achieving the highest accuracy when provided with precise anatomical references.
  arXiv  Detail & Related papers  (2025-04-30T19:01:06Z)
• Comparative Evaluation of Radiomics and Deep Learning Models for Disease Detection in Chest Radiography [0.0]
  This study presents a comprehensive evaluation of radiomics-based and deep learning-based approaches for disease detection in chest radiography. It focuses on COVID-19, lung opacity, and viral pneumonia. The results aim to inform the integration of AI-driven diagnostic tools in clinical practice.
  arXiv  Detail & Related papers  (2025-04-16T16:54:37Z)
• Advancing Chronic Tuberculosis Diagnostics Using Vision-Language Models: A Multi modal Framework for Precision Analysis [0.0]
  Vision-Language Model (VLM) uses a Vision Transformer (ViT) for visual encoding and a transformer-based text encoder to process clinical context.<n>Cross-modal attention mechanisms align radiographic features with textual information, while the Gemma-3b decoder generates comprehensive diagnostic reports.<n>Model demonstrated high precision (94 percent) and recall (94 percent) for detecting key chronic TB pathologies.
  arXiv  Detail & Related papers  (2025-03-17T13:49:29Z)
• Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Clinical Pathology Analysis [34.199766079609795]
  Pathological diagnosis is vital for determining disease characteristics, guiding treatment, and assessing prognosis.<n>Traditional pure vision models face challenges of redundant feature extraction.<n>Existing large vision-language models (LVLMs) are limited by input resolution constraints, hindering their efficiency and accuracy.<n>We propose two innovative strategies: the mixed task-guided feature enhancement, and the prompt-guided detail feature completion.
  arXiv  Detail & Related papers  (2024-12-12T18:07:23Z)
• MvKeTR: Chest CT Report Generation with Multi-View Perception and Knowledge Enhancement [1.4680538148112467]
  Multi-view perception knowledge-enhanced Transformer (MvKeTR)<n>MVPA with view-aware attention effectively synthesizes diagnostic information from multiple anatomical views.<n>Cross-Modal Knowledge Enhancer (CMKE) retrieves the most similar reports based on the query volume.
  arXiv  Detail & Related papers  (2024-11-27T12:58:23Z)
• Enhancing Osteoporosis Detection: An Explainable Multi-Modal Learning Framework with Feature Fusion and Variable Clustering [6.196283036344105]
  Osteoporosis is a common condition that increases fracture risk, especially in older adults. This study presents a novel multi-modal learning framework that integrates clinical and imaging data to improve diagnostic accuracy and model interpretability.
  arXiv  Detail & Related papers  (2024-11-01T13:58:15Z)
• Super-resolution of biomedical volumes with 2D supervision [84.5255884646906]
  Masked slice diffusion for super-resolution exploits the inherent equivalence in the data-generating distribution across all spatial dimensions of biological specimens. We focus on the application of SliceR to stimulated histology (SRH), characterized by its rapid acquisition of high-resolution 2D images but slow and costly optical z-sectioning.
  arXiv  Detail & Related papers  (2024-04-15T02:41:55Z)
• Revisiting Computer-Aided Tuberculosis Diagnosis [56.80999479735375]
  Tuberculosis (TB) is a major global health threat, causing millions of deaths annually. Computer-aided tuberculosis diagnosis (CTD) using deep learning has shown promise, but progress is hindered by limited training data. We establish a large-scale dataset, namely the Tuberculosis X-ray (TBX11K) dataset, which contains 11,200 chest X-ray (CXR) images with corresponding bounding box annotations for TB areas. This dataset enables the training of sophisticated detectors for high-quality CTD.
  arXiv  Detail & Related papers  (2023-07-06T08:27:48Z)
• A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics [63.106382317917344]
  We report a Transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary diseases.
  arXiv  Detail & Related papers  (2023-06-01T16:23:47Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• Diagnosis of Coronavirus Disease 2019 (COVID-19) with Structured Latent Multi-View Representation Learning [48.05232274463484]
  Recently, the outbreak of Coronavirus Disease 2019 (COVID-19) has spread rapidly across the world. Due to the large number of affected patients and heavy labor for doctors, computer-aided diagnosis with machine learning algorithm is urgently needed. In this study, we propose to conduct the diagnosis of COVID-19 with a series of features extracted from CT images.
  arXiv  Detail & Related papers  (2020-05-06T15:19:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.