When Eye-Tracking Meets Machine Learning: A Systematic Review on
Applications in Medical Image Analysis
- URL: http://arxiv.org/abs/2403.07834v1
- Date: Tue, 12 Mar 2024 17:17:20 GMT
- Title: When Eye-Tracking Meets Machine Learning: A Systematic Review on
Applications in Medical Image Analysis
- Authors: Sahar Moradizeyveh, Mehnaz Tabassum, Sidong Liu, Robert Ahadizad
Newport, Amin Beheshti, Antonio Di Ieva
- Abstract summary: Eye tracking, a technology that monitors and records the movement of the eyes, provides valuable insights into human visual attention patterns.
Eye-gaze tracking data, with intricate human visual attention patterns embedded, provides a bridge to integrating artificial intelligence (AI) development and human cognition.
This systematic review investigates eye-gaze tracking applications and methodologies for enhancing ML/DL algorithms for medical image analysis in depth.
- Score: 2.9122893700072554
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Eye-gaze tracking research offers significant promise in enhancing various
healthcare-related tasks, above all in medical image analysis and
interpretation. Eye tracking, a technology that monitors and records the
movement of the eyes, provides valuable insights into human visual attention
patterns. This technology can transform how healthcare professionals and
medical specialists engage with and analyze diagnostic images, offering a more
insightful and efficient approach to medical diagnostics. Hence, extracting
meaningful features and insights from medical images by leveraging eye-gaze
data improves our understanding of how radiologists and other medical experts
monitor, interpret, and understand images for diagnostic purposes. Eye-tracking
data, with intricate human visual attention patterns embedded, provides a
bridge to integrating artificial intelligence (AI) development and human
cognition. This integration allows novel methods to incorporate domain
knowledge into machine learning (ML) and deep learning (DL) approaches to
enhance their alignment with human-like perception and decision-making.
Moreover, extensive collections of eye-tracking data have also enabled novel
ML/DL methods to analyze human visual patterns, paving the way to a better
understanding of human vision, attention, and cognition. This systematic review
investigates eye-gaze tracking applications and methodologies for enhancing
ML/DL algorithms for medical image analysis in depth.
Related papers
- Neural 3D decoding for human vision diagnosis [76.41771117405973]
We show how AI can go beyond the current state of the art by advancing from 2D visuals to visually plausible and functionally more comprehensive 3D visuals decoded from brain signals.
We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject who was presented with a 2D image, and yields as output the corresponding 3D object visuals.
arXiv Detail & Related papers (2024-05-24T06:06:11Z) - Using artificial intelligence methods for the studyed visual analyzer [0.0]
The paper describes how various techniques for applying artificial intelligence to the study of human eyes are utilized.
The first dataset was collected using computerized perimetry to investigate the visualization of the human visual field and the diagnosis of glaucoma.
The second dataset was obtained, as part of the implementation of a Russian-Swiss experiment to collect and analyze eye movement data using the Tobii Pro Glasses 3 device on VR video.
arXiv Detail & Related papers (2024-04-25T20:12:51Z) - MLIP: Enhancing Medical Visual Representation with Divergence Encoder
and Knowledge-guided Contrastive Learning [48.97640824497327]
We propose a novel framework leveraging domain-specific medical knowledge as guiding signals to integrate language information into the visual domain through image-text contrastive learning.
Our model includes global contrastive learning with our designed divergence encoder, local token-knowledge-patch alignment contrastive learning, and knowledge-guided category-level contrastive learning with expert knowledge.
Notably, MLIP surpasses state-of-the-art methods even with limited annotated data, highlighting the potential of multimodal pre-training in advancing medical representation learning.
arXiv Detail & Related papers (2024-02-03T05:48:50Z) - Deep Learning and Computer Vision for Glaucoma Detection: A Review [0.8379286663107844]
Glaucoma is the leading cause of irreversible blindness worldwide.
Recent advances in computer vision and deep learning have demonstrated the potential for automated assessment.
We survey recent studies on AI-based glaucoma diagnosis using fundus, optical coherence tomography, and visual field images.
arXiv Detail & Related papers (2023-07-31T09:49:51Z) - Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges [58.32937972322058]
"Medico automatic polyp segmentation (Medico 2020)" and "MedAI: Transparency in Medical Image (MedAI 2021)" competitions.
We present a comprehensive summary and analyze each contribution, highlight the strength of the best-performing methods, and discuss the possibility of clinical translations of such methods into the clinic.
arXiv Detail & Related papers (2023-07-30T16:08:45Z) - Align, Reason and Learn: Enhancing Medical Vision-and-Language
Pre-training with Knowledge [68.90835997085557]
We propose a systematic and effective approach to enhance structured medical knowledge from three perspectives.
First, we align the representations of the vision encoder and the language encoder through knowledge.
Second, we inject knowledge into the multi-modal fusion model to enable the model to perform reasoning using knowledge as the supplementation of the input image and text.
Third, we guide the model to put emphasis on the most critical information in images and texts by designing knowledge-induced pretext tasks.
arXiv Detail & Related papers (2022-09-15T08:00:01Z) - Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis [54.60796004113496]
We demonstrate that the eye movement of radiologists reading medical images can be a new form of supervision to train the DNN-based computer-aided diagnosis (CAD) system.
We record the tracks of the radiologists' gaze when they are reading images.
The gaze information is processed and then used to supervise the DNN's attention via an Attention Consistency module.
arXiv Detail & Related papers (2022-04-06T08:31:05Z) - Self-supervised learning methods and applications in medical imaging
analysis: A survey [0.0]
The article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with concentration on their applications in the field of medical imaging analysis.
The article covers (40) of the most recent researches in the field of self-supervised learning in medical imaging analysis aiming at shedding the light on the recent innovation in the field.
arXiv Detail & Related papers (2021-09-17T17:01:42Z) - Recent advances and clinical applications of deep learning in medical
image analysis [7.132678647070632]
We reviewed and summarized more than 200 recently published papers to provide a comprehensive overview of applying deep learning methods in various medical image analysis tasks.
Especially, we emphasize the latest progress and contributions of state-of-the-art unsupervised and semi-supervised deep learning in medical images.
arXiv Detail & Related papers (2021-05-27T18:05:12Z) - An Interpretable Multiple-Instance Approach for the Detection of
referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images.
By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy.
We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
arXiv Detail & Related papers (2021-03-02T13:14:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.