GazeSearch: Radiology Findings Search Benchmark

• URL: http://arxiv.org/abs/2411.05780v2
• Date: Wed, 27 Nov 2024 19:01:53 GMT
• Title: GazeSearch: Radiology Findings Search Benchmark
• Authors: Trong Thang Pham, Tien-Phat Nguyen, Yuki Ikebe, Akash Awasthi, Zhigang Deng, Carol C. Wu, Hien Nguyen, Ngan Le,
• Abstract summary: Medical eye-tracking data is an important information source for understanding how radiologists visually interpret medical images. The current eye-tracking data is dispersed, unprocessed, and ambiguous, making it difficult to derive meaningful insights. In this work, we propose a refinement method inspired by the target-present visual search challenge: there is a specific finding and fixations are guided to locate it.
• Score: 9.21918773048464
• License:
• Abstract: Medical eye-tracking data is an important information source for understanding how radiologists visually interpret medical images. This information not only improves the accuracy of deep learning models for X-ray analysis but also their interpretability, enhancing transparency in decision-making. However, the current eye-tracking data is dispersed, unprocessed, and ambiguous, making it difficult to derive meaningful insights. Therefore, there is a need to create a new dataset with more focus and purposeful eyetracking data, improving its utility for diagnostic applications. In this work, we propose a refinement method inspired by the target-present visual search challenge: there is a specific finding and fixations are guided to locate it. After refining the existing eye-tracking datasets, we transform them into a curated visual search dataset, called GazeSearch, specifically for radiology findings, where each fixation sequence is purposefully aligned to the task of locating a particular finding. Subsequently, we introduce a scan path prediction baseline, called ChestSearch, specifically tailored to GazeSearch. Finally, we employ the newly introduced GazeSearch as a benchmark to evaluate the performance of current state-of-the-art methods, offering a comprehensive assessment for visual search in the medical imaging domain. Code is available at \url{https://github.com/UARK-AICV/GazeSearch}.

Related papers

• Eye-gaze Guided Multi-modal Alignment for Medical Representation Learning [65.54680361074882]
  Eye-gaze Guided Multi-modal Alignment (EGMA) framework harnesses eye-gaze data for better alignment of medical visual and textual features. We conduct downstream tasks of image classification and image-text retrieval on four medical datasets.
  arXiv  Detail & Related papers  (2024-03-19T03:59:14Z)
• I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses [9.260958560874812]
  Interpretable Artificial Intelligence (I-AI) is a novel and unified controllable interpretable pipeline. Our I-AI addresses three key questions: where a radiologist looks, how long they focus on specific areas, and what findings they diagnose.
  arXiv  Detail & Related papers  (2023-09-24T04:48:44Z)
• Deep Learning and Computer Vision for Glaucoma Detection: A Review [0.8379286663107844]
  Glaucoma is the leading cause of irreversible blindness worldwide. Recent advances in computer vision and deep learning have demonstrated the potential for automated assessment. We survey recent studies on AI-based glaucoma diagnosis using fundus, optical coherence tomography, and visual field images.
  arXiv  Detail & Related papers  (2023-07-31T09:49:51Z)
• GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray Classification [9.266556662553345]
  We propose a novel gaze-guided graph neural network (GNN), GazeGNN, to leverage raw eye-gaze data without being converted into visual attention maps (VAMs) We develop a real-time, real-world, end-to-end disease classification algorithm for the first time in the literature.
  arXiv  Detail & Related papers  (2023-05-29T17:01:54Z)
• Robust Detection Outcome: A Metric for Pathology Detection in Medical Images [6.667150890634173]
  Robust Detection Outcome (RoDeO) is a novel metric for evaluating algorithms for pathology detection in medical images. RoDeO evaluates different errors directly and individually, and reflects clinical needs better than current metrics.
  arXiv  Detail & Related papers  (2023-03-03T13:45:13Z)
• Data-Efficient Vision Transformers for Multi-Label Disease Classification on Chest Radiographs [55.78588835407174]
  Vision Transformers (ViTs) have not been applied to this task despite their high classification performance on generic images. ViTs do not rely on convolutions but on patch-based self-attention and in contrast to CNNs, no prior knowledge of local connectivity is present. Our results show that while the performance between ViTs and CNNs is on par with a small benefit for ViTs, DeiTs outperform the former if a reasonably large data set is available for training.
  arXiv  Detail & Related papers  (2022-08-17T09:07:45Z)
• Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis [17.0847996323416]
  We propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectively strengthens the discriminability of hash codes. Benefit from this, the superfluous information is reduced, which facilitates the discriminability of hash codes. Experimental results demonstrate the superior accuracy of the proposed DSIBH compared with state-of-the-arts in cross-modal medical data retrieval tasks.
  arXiv  Detail & Related papers  (2022-05-06T11:43:17Z)
• Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis [54.60796004113496]
  We demonstrate that the eye movement of radiologists reading medical images can be a new form of supervision to train the DNN-based computer-aided diagnosis (CAD) system. We record the tracks of the radiologists' gaze when they are reading images. The gaze information is processed and then used to supervise the DNN's attention via an Attention Consistency module.
  arXiv  Detail & Related papers  (2022-04-06T08:31:05Z)
• Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Data [72.1187887376849]
  The selective attention mechanism helps the cognition system focus on task-relevant visual clues by ignoring the presence of distractors. We propose a framework to leverage gaze for medical image analysis tasks with small training data. Our method is demonstrated to achieve superior performance on both 3D tumor segmentation and 2D chest X-ray classification tasks.
  arXiv  Detail & Related papers  (2021-12-02T07:55:25Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)
• Learning Invariant Feature Representation to Improve Generalization across Chest X-ray Datasets [55.06983249986729]
  We show that a deep learning model performing well when tested on the same dataset as training data starts to perform poorly when it is tested on a dataset from a different source. By employing an adversarial training strategy, we show that a network can be forced to learn a source-invariant representation.
  arXiv  Detail & Related papers  (2020-08-04T07:41:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.