Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis
- URL: http://arxiv.org/abs/2204.02976v1
- Date: Wed, 6 Apr 2022 08:31:05 GMT
- Title: Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis
- Authors: Sheng Wang, Xi Ouyang, Tianming Liu, Qian Wang, Dinggang Shen
- Abstract summary: We demonstrate that the eye movement of radiologists reading medical images can be a new form of supervision to train the DNN-based computer-aided diagnosis (CAD) system.
We record the tracks of the radiologists' gaze when they are reading images.
The gaze information is processed and then used to supervise the DNN's attention via an Attention Consistency module.
- Score: 54.60796004113496
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: When deep neural network (DNN) was first introduced to the medical image
analysis community, researchers were impressed by its performance. However, it
is evident now that a large number of manually labeled data is often a must to
train a properly functioning DNN. This demand for supervision data and labels
is a major bottleneck in current medical image analysis, since collecting a
large number of annotations from experienced experts can be time-consuming and
expensive. In this paper, we demonstrate that the eye movement of radiologists
reading medical images can be a new form of supervision to train the DNN-based
computer-aided diagnosis (CAD) system. Particularly, we record the tracks of
the radiologists' gaze when they are reading images. The gaze information is
processed and then used to supervise the DNN's attention via an Attention
Consistency module. To the best of our knowledge, the above pipeline is among
the earliest efforts to leverage expert eye movement for deep-learning-based
CAD. We have conducted extensive experiments on knee X-ray images for
osteoarthritis assessment. The results show that our method can achieve
considerable improvement in diagnosis performance, with the help of gaze
supervision.
Related papers
- Gaze-directed Vision GNN for Mitigating Shortcut Learning in Medical Image [6.31072075551707]
We propose a novel gaze-directed Vision GNN (called GD-ViG) to leverage the visual patterns of radiologists from gaze as expert knowledge.
The experiments on two public medical image datasets demonstrate that GD-ViG outperforms the state-of-the-art methods.
arXiv Detail & Related papers (2024-06-20T07:16:41Z) - Object Detection for Automated Coronary Artery Using Deep Learning [0.0]
In our paper, we utilize the object detection method on X-ray angiography images to precisely identify the location of coronary artery stenosis.
This model enables automatic and real-time detection of stenosis locations, assisting in the crucial and sensitive decision-making process.
arXiv Detail & Related papers (2023-12-19T13:14:52Z) - Case Studies on X-Ray Imaging, MRI and Nuclear Imaging [0.0]
We will focus on how AI-based approaches, particularly the use of Convolutional Neural Networks (CNN), can assist in disease detection through medical imaging technology.
CNN is a commonly used approach for image analysis due to its ability to extract features from raw input images.
arXiv Detail & Related papers (2023-06-03T09:05:35Z) - GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray
Classification [9.266556662553345]
We propose a novel gaze-guided graph neural network (GNN), GazeGNN, to leverage raw eye-gaze data without being converted into visual attention maps (VAMs)
We develop a real-time, real-world, end-to-end disease classification algorithm for the first time in the literature.
arXiv Detail & Related papers (2023-05-29T17:01:54Z) - Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report
Generation [92.73584302508907]
We propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning.
In detail, the fundamental structure of our graph is pre-constructed from general knowledge.
Each image feature is integrated with its very own updated graph before being fed into the decoder module for report generation.
arXiv Detail & Related papers (2023-03-18T03:53:43Z) - A Deep Learning Approach for the Segmentation of Electroencephalography
Data in Eye Tracking Applications [56.458448869572294]
We introduce DETRtime, a novel framework for time-series segmentation of EEG data.
Our end-to-end deep learning-based framework brings advances in Computer Vision to the forefront.
Our model generalizes well in the task of EEG sleep stage segmentation.
arXiv Detail & Related papers (2022-06-17T10:17:24Z) - Leveraging Human Selective Attention for Medical Image Analysis with
Limited Training Data [72.1187887376849]
The selective attention mechanism helps the cognition system focus on task-relevant visual clues by ignoring the presence of distractors.
We propose a framework to leverage gaze for medical image analysis tasks with small training data.
Our method is demonstrated to achieve superior performance on both 3D tumor segmentation and 2D chest X-ray classification tasks.
arXiv Detail & Related papers (2021-12-02T07:55:25Z) - Voice-assisted Image Labelling for Endoscopic Ultrasound Classification
using Neural Networks [48.732863591145964]
We propose a multi-modal convolutional neural network architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure.
Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels.
arXiv Detail & Related papers (2021-10-12T21:22:24Z) - Anomaly Detection in Medical Imaging -- A Mini Review [0.8122270502556374]
This paper uses a semi-exhaustive literature review of relevant anomaly detection papers in medical imaging to cluster into applications.
The main results showed that the current research is mostly motivated by reducing the need for labelled data.
Also, the successful and substantial amount of research in the brain MRI domain shows the potential for applications in further domains like OCT and chest X-ray.
arXiv Detail & Related papers (2021-08-25T11:45:40Z) - An Interpretable Multiple-Instance Approach for the Detection of
referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images.
By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy.
We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
arXiv Detail & Related papers (2021-03-02T13:14:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.