Sketch-based Medical Image Retrieval

• URL: http://arxiv.org/abs/2303.03633v1
• Date: Tue, 7 Mar 2023 03:41:13 GMT
• Title: Sketch-based Medical Image Retrieval
• Abstract summary: We introduce a novel sketch-based medical image retrieval system that enables users to find images of interest without example images. Key idea lies in feature decomposition of medical images, whereby the entire feature of a medical image can be decomposed into and reconstructed from normal and abnormal features. Our system achieves flexible medical image retrieval on demand, thereby expanding the utility of medical image databases.
• Score: 37.009806114204636
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The amount of medical images stored in hospitals is increasing faster than ever; however, utilizing the accumulated medical images has been limited. This is because existing content-based medical image retrieval (CBMIR) systems usually require example images to construct query vectors; nevertheless, example images cannot always be prepared. Besides, there can be images with rare characteristics that make it difficult to find similar example images, which we call isolated samples. Here, we introduce a novel sketch-based medical image retrieval (SBMIR) system that enables users to find images of interest without example images. The key idea lies in feature decomposition of medical images, whereby the entire feature of a medical image can be decomposed into and reconstructed from normal and abnormal features. By extending this idea, our SBMIR system provides an easy-to-use two-step graphical user interface: users first select a template image to specify a normal feature and then draw a semantic sketch of the disease on the template image to represent an abnormal feature. Subsequently, it integrates the two kinds of input to construct a query vector and retrieves reference images with the closest reference vectors. Using two datasets, ten healthcare professionals with various clinical backgrounds participated in the user test for evaluation. As a result, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for isolated samples. Our SBMIR system achieves flexible medical image retrieval on demand, thereby expanding the utility of medical image databases.

Related papers

• MIRAGE: Retrieval and Generation of Multimodal Images and Texts for Medical Education [2.765006963244799]
  MIRAGE is a multimodal medical text and image retrieval and generation system.<n>It allows users to find and generate clinically relevant images from trustworthy sources.<n>The system is accessible to medical students worldwide without requiring local computational resources or technical expertise.
  arXiv  Detail & Related papers  (2026-05-06T11:20:08Z)
• RadIR: A Scalable Framework for Multi-Grained Medical Image Retrieval via Radiology Report Mining [48.21287619304126]
  We propose a novel methodology that leverages dense radiology reports to define image-wise similarity ordering at multiple granularities. We construct two comprehensive medical imaging retrieval datasets: MIMIC-IR for Chest X-rays and CTRATE-IR for CT scans. We develop two retrieval systems, RadIR-CXR and model-ChestCT, which demonstrate superior performance in traditional image-image and image-report retrieval tasks.
  arXiv  Detail & Related papers  (2025-03-06T17:43:03Z)
• Fréchet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets [13.737058479403311]
  We introduce a new perceptual metric tailored for medical images, FRD (Fr'echet Radiomic Distance)<n>We show that FRD is superior to other image distribution metrics for a range of medical imaging applications.<n> FRD offers additional benefits such as stability and computational efficiency at low sample sizes.
  arXiv  Detail & Related papers  (2024-12-02T13:49:14Z)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
  We introduce the Med-ST framework for fine-grained spatial and temporal modeling. For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views. For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
  arXiv  Detail & Related papers  (2024-05-30T03:15:09Z)
• Understanding differences in applying DETR to natural and medical images [16.200340490559338]
  Transformer-based detectors have shown success in computer vision tasks with natural images. Medical imaging data presents unique challenges such as extremely large image sizes, fewer and smaller regions of interest, and object classes which can be differentiated only through subtle differences. This study evaluates the applicability of these transformer-based design choices when applied to a screening mammography dataset.
  arXiv  Detail & Related papers  (2024-05-27T22:06:42Z)
• Benchmarking Pretrained Vision Embeddings for Near- and Duplicate Detection in Medical Images [0.6827423171182154]
  We present an approach for identifying near- and duplicate 3D medical images leveraging publicly available 2D computer vision embeddings. We generate an experimental benchmark based on the publicly available Medical Decathlon dataset.
  arXiv  Detail & Related papers  (2023-12-12T13:52:55Z)
• Few-shot Medical Image Segmentation with Cycle-resemblance Attention [20.986884555902183]
  Few-shot learning has gained increasing attention in the medical image semantic segmentation field. In this paper, we propose a novel self-supervised few-shot medical image segmentation network. We introduce a novel Cycle-Resemblance Attention (CRA) module to fully leverage the pixel-wise relation between query and support medical images.
  arXiv  Detail & Related papers  (2022-12-07T21:55:26Z)
• Multimorbidity Content-Based Medical Image Retrieval Using Proxies [37.47987844057842]
  We propose a novel multi-label metric learning method that can be used for both classification and content-based image retrieval. Our model is able to support diagnosis by predicting the presence of diseases and provide evidence for these predictions. We demonstrate the efficacy of our approach to both classification and content-based image retrieval on two multimorbidity radiology datasets.
  arXiv  Detail & Related papers  (2022-11-22T11:23:53Z)
• Proxy-bridged Image Reconstruction Network for Anomaly Detection in Medical Images [59.700111685673846]
  Anomaly detection in medical images refers to the identification of abnormal images with only normal images in the training set. We propose a novel Proxy-bridged Image Reconstruction Network ( ProxyAno) for anomaly detection in medical images.
  arXiv  Detail & Related papers  (2021-10-05T00:40:43Z)
• Generative Adversarial U-Net for Domain-free Medical Image Augmentation [49.72048151146307]
  The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. In this paper, we develop a novel generative method named generative adversarial U-Net. Our newly designed model is domain-free and generalizable to various medical images.
  arXiv  Detail & Related papers  (2021-01-12T23:02:26Z)
• Convolutional-LSTM for Multi-Image to Single Output Medical Prediction [55.41644538483948]
  A common scenario in developing countries is to have the volume metadata lost due multiple reasons. It is possible to get a multi-image to single diagnostic model which mimics human doctor diagnostic process.
  arXiv  Detail & Related papers  (2020-10-20T04:30:09Z)
• A Bag of Visual Words Model for Medical Image Retrieval [0.9137554315375919]
  Bag of Visual Words (BoVW) is a technique that can be used to effectively represent intrinsic image features in vector space. We present a MedIR approach based on the BoVW model for content-based medical image retrieval.
  arXiv  Detail & Related papers  (2020-07-18T16:21:30Z)
• Universal Model for Multi-Domain Medical Image Retrieval [88.67940265012638]
  Medical Image Retrieval (MIR) helps doctors quickly find similar patients' data. MIR is becoming increasingly helpful due to the wide use of digital imaging modalities. However, the popularity of various digital imaging modalities in hospitals also poses several challenges to MIR.
  arXiv  Detail & Related papers  (2020-07-14T23:22:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.