A clinically motivated self-supervised approach for content-based image retrieval of CT liver images

• URL: http://arxiv.org/abs/2207.04812v1
• Date: Mon, 11 Jul 2022 12:16:29 GMT
• Title: A clinically motivated self-supervised approach for content-based image retrieval of CT liver images
• Authors: Kristoffer Knutsen Wickstr{\o}m and Eirik Agnalt {\O}stmo and Keyur Radiya and Karl {\O}yvind Mikalsen and Michael Christian Kampffmeyer and Robert Jenssen
• Abstract summary: We propose a self-supervised learning framework that incorporates domain-knowledge into the training procedure. We also provide the first representation learning explainability analysis in the context of CBIR of CT liver images. Results demonstrate improved performance compared to the standard self-supervised approach.
• Score: 11.829945671246598
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning-based approaches for content-based image retrieval (CBIR) of CT liver images is an active field of research, but suffers from some critical limitations. First, they are heavily reliant on labeled data, which can be challenging and costly to acquire. Second, they lack transparency and explainability, which limits the trustworthiness of deep CBIR systems. We address these limitations by (1) proposing a self-supervised learning framework that incorporates domain-knowledge into the training procedure and (2) providing the first representation learning explainability analysis in the context of CBIR of CT liver images. Results demonstrate improved performance compared to the standard self-supervised approach across several metrics, as well as improved generalisation across datasets. Further, we conduct the first representation learning explainability analysis in the context of CBIR, which reveals new insights into the feature extraction process. Lastly, we perform a case study with cross-examination CBIR that demonstrates the usability of our proposed framework. We believe that our proposed framework could play a vital role in creating trustworthy deep CBIR systems that can successfully take advantage of unlabeled data.

Related papers

• Mind the Context: Attention-Guided Weak-to-Strong Consistency for Enhanced Semi-Supervised Medical Image Segmentation [14.67636369741001]
  This paper introduces a semi-supervised learning framework named Attention-Guided weak-to-strong Consistency Match (AIGCMatch) The AIGCMatch framework incorporates attention-guided perturbation strategies at both the image and feature levels to achieve weak-to-strong consistency regularization. Our method achieved a 90.4% Dice score in the 7-case scenario on the ACDC dataset, surpassing the state-of-the-art methods and demonstrating its potential and efficacy in clinical settings.
  arXiv  Detail & Related papers  (2024-10-16T10:04:22Z)
• Advancing Medical Radiograph Representation Learning: A Hybrid Pre-training Paradigm with Multilevel Semantic Granularity [14.223539927549782]
  We propose a novel HybridMED framework to align global-level visual representations with impression and token-level visual representations with findings. Our framework incorporates a generation decoder that employs two proxy tasks, responsible for generating the impression from images, via a captioning branch, and (2) findings, through a summarization branch. Experiments on the MIMIC-CXR dataset reveal that our summarization branch effectively distills knowledge to the captioning branch, enhancing model performance without significantly increasing parameter requirements.
  arXiv  Detail & Related papers  (2024-10-01T07:05:36Z)
• Symmetrical Bidirectional Knowledge Alignment for Zero-Shot Sketch-Based Image Retrieval [69.46139774646308]
  This paper studies the problem of zero-shot sketch-based image retrieval (ZS-SBIR) It aims to use sketches from unseen categories as queries to match the images of the same category. We propose a novel Symmetrical Bidirectional Knowledge Alignment for zero-shot sketch-based image retrieval (SBKA)
  arXiv  Detail & Related papers  (2023-12-16T04:50:34Z)
• Advancements in Content-Based Image Retrieval: A Comprehensive Survey of Relevance Feedback Techniques [0.0]
  Content-based image retrieval (CBIR) systems have emerged as crucial tools in the field of computer vision. This survey paper presents a comprehensive overview of CBIR, emphasizing its role in object detection and its potential to identify and retrieve visually similar images based on content features. It elaborates on the semantic gap, which arises from the disparity between low-level features and high-level semantic concepts, and explores approaches to bridge this gap.
  arXiv  Detail & Related papers  (2023-12-13T11:07:32Z)
• Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition [57.08108545219043]
  Recognizing novel sub-categories with scarce samples is an essential and challenging research topic in computer vision. Existing literature addresses this challenge by employing local-based representation approaches. This article proposes a novel model, Robust Saliency-aware Distillation (RSaD), for few-shot fine-grained visual recognition.
  arXiv  Detail & Related papers  (2023-05-12T00:13:17Z)
• Localized Region Contrast for Enhancing Self-Supervised Learning in Medical Image Segmentation [27.82940072548603]
  We propose a novel contrastive learning framework that integrates Localized Region Contrast (LRC) to enhance existing self-supervised pre-training methods for medical image segmentation. Our approach involves identifying Super-pixels by Felzenszwalb's algorithm and performing local contrastive learning using a novel contrastive sampling loss.
  arXiv  Detail & Related papers  (2023-04-06T22:43:13Z)
• A Knowledge-based Learning Framework for Self-supervised Pre-training Towards Enhanced Recognition of Medical Images [14.304996977665212]
  This study proposes a knowledge-based learning framework towards enhanced recognition of medical images. It works in three phases by synergizing contrastive learning and generative learning models. The proposed framework statistically excels in self-supervised benchmarks, achieving 2.08, 1.23, 1.12, 0.76 and 1.38 percentage points improvements over SimCLR in AUC/Dice.
  arXiv  Detail & Related papers  (2022-11-27T03:58:58Z)
• A Principled Design of Image Representation: Towards Forensic Tasks [75.40968680537544]
  We investigate the forensic-oriented image representation as a distinct problem, from the perspectives of theory, implementation, and application. At the theoretical level, we propose a new representation framework for forensics, called Dense Invariant Representation (DIR), which is characterized by stable description with mathematical guarantees. We demonstrate the above arguments on the dense-domain pattern detection and matching experiments, providing comparison results with state-of-the-art descriptors.
  arXiv  Detail & Related papers  (2022-03-02T07:46:52Z)
• Towards Unbiased COVID-19 Lesion Localisation and Segmentation via Weakly Supervised Learning [66.36706284671291]
  We propose a data-driven framework supervised by only image-level labels to support unbiased lesion localisation. The framework can explicitly separate potential lesions from original images, with the help of a generative adversarial network and a lesion-specific decoder.
  arXiv  Detail & Related papers  (2021-03-01T06:05:49Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)
• Self-supervised Learning from a Multi-view Perspective [121.63655399591681]
  We show that self-supervised representations can extract task-relevant information and discard task-irrelevant information. Our theoretical framework paves the way to a larger space of self-supervised learning objective design.
  arXiv  Detail & Related papers  (2020-06-10T00:21:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.