Medical Image Retrieval Using Pretrained Embeddings

• URL: http://arxiv.org/abs/2311.13547v1
• Date: Wed, 22 Nov 2023 17:42:33 GMT
• Title: Medical Image Retrieval Using Pretrained Embeddings
• Authors: Farnaz Khun Jush, Tuan Truong, Steffen Vogler, Matthias Lenga
• Abstract summary: We show that medical image retrieval is feasible using pretrained networks without any additional training or fine-tuning steps. Using pretrained embeddings, we achieved a recall of 1 for various tasks at modality, body region, and organ level.
• Score: 0.6827423171182154
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A wide range of imaging techniques and data formats available for medical images make accurate retrieval from image databases challenging. Efficient retrieval systems are crucial in advancing medical research, enabling large-scale studies and innovative diagnostic tools. Thus, addressing the challenges of medical image retrieval is essential for the continued enhancement of healthcare and research. In this study, we evaluated the feasibility of employing four state-of-the-art pretrained models for medical image retrieval at modality, body region, and organ levels and compared the results of two similarity indexing approaches. Since the employed networks take 2D images, we analyzed the impacts of weighting and sampling strategies to incorporate 3D information during retrieval of 3D volumes. We showed that medical image retrieval is feasible using pretrained networks without any additional training or fine-tuning steps. Using pretrained embeddings, we achieved a recall of 1 for various tasks at modality, body region, and organ level.

Related papers

• Autoregressive Sequence Modeling for 3D Medical Image Representation [48.706230961589924]
  We introduce a pioneering method for learning 3D medical image representations through an autoregressive sequence pre-training framework. Our approach various 3D medical images based on spatial, contrast, and semantic correlations, treating them as interconnected visual tokens within a token sequence.
  arXiv  Detail & Related papers  (2024-09-13T10:19:10Z)
• Content-Based Image Retrieval for Multi-Class Volumetric Radiology Images: A Benchmark Study [0.6249768559720122]
  We benchmark embeddings derived from pre-trained supervised models on medical images against embeddings derived from pre-trained unsupervised models on non-medical images. For volumetric image retrieval, we adopt a late interaction re-ranking method inspired by text matching.
  arXiv  Detail & Related papers  (2024-05-15T13:34:07Z)
• BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval [44.92177279141073]
  We present a dataset of 8,069 3D CT volumes, encompassing over 2 million slices, paired with their respective radiological reports. We then craft a retrieval strategy, MedFinder, harnessing the potential of large language models. It marks our preliminary step towards developing a system capable of facilitating text-to-image, image-to-text, and keyword-based retrieval tasks.
  arXiv  Detail & Related papers  (2024-03-24T03:10:07Z)
• Unified Medical Image Pre-training in Language-Guided Common Semantic Space [39.61770813855078]
  We propose an Unified Medical Image Pre-training framework, namely UniMedI. UniMedI uses diagnostic reports as common semantic space to create unified representations for diverse modalities of medical images. We evaluate its performance on both 2D and 3D images across 10 different datasets.
  arXiv  Detail & Related papers  (2023-11-24T22:01:12Z)
• 3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology [6.851500027718433]
  The field of 3D medical image retrieval is still emerging, lacking established evaluation benchmarks, comprehensive datasets, and thorough studies. This paper introduces a novel benchmark for 3D Medical Image Retrieval (3D-MIR) that encompasses four different anatomies imaged with computed tomography. Using this benchmark, we explore a diverse set of search strategies that use aggregated 2D slices, 3D volumes, and multi-modal embeddings from popular multi-modal foundation models as queries.
  arXiv  Detail & Related papers  (2023-11-23T00:57:35Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions [66.40971096248946]
  In this paper, we collect a series of MedISeg tricks for different model implementation phases. We experimentally explore the effectiveness of these tricks on consistent baselines. We also open-sourced a strong MedISeg repository, where each component has the advantage of plug-and-play.
  arXiv  Detail & Related papers  (2022-09-21T12:30:05Z)
• Medical Image Segmentation using 3D Convolutional Neural Networks: A Review [25.864941088823343]
  Computer-aided medical image analysis plays a significant role in assisting medical practitioners for expert clinical diagnosis and deciding the optimal treatment plan. At present, convolutional neural networks (CNN) are the preferred choice for medical image analysis. With the rapid advancements in 3D imaging systems and the availability of excellent hardware and software support, 3D deep learning methods are gaining popularity in medical image analysis.
  arXiv  Detail & Related papers  (2021-08-19T03:23:08Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Learning Binary Semantic Embedding for Histology Image Classification and Retrieval [56.34863511025423]
  We propose a novel method for Learning Binary Semantic Embedding (LBSE) Based on the efficient and effective embedding, classification and retrieval are performed to provide interpretable computer-assisted diagnosis for histology images. Experiments conducted on three benchmark datasets validate the superiority of LBSE under various scenarios.
  arXiv  Detail & Related papers  (2020-10-07T08:36:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.