Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models

• URL: http://arxiv.org/abs/2304.13536v1
• Date: Wed, 12 Apr 2023 10:15:31 GMT
• Title: Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models
• Authors: Daniel G. Krakowczyk, Paul Prasse, David R. Reich, Sebastian Lapuschkin, Tobias Scheffer, Lena A. J\"ager
• Abstract summary: In this work, we employ established gaze event detection algorithms for fixations and saccades. We quantitatively evaluate the impact of these events by determining their concept influence.
• Score: 0.7829352305480283
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work in XAI for eye tracking data has evaluated the suitability of feature attribution methods to explain the output of deep neural sequence models for the task of oculomotric biometric identification. These methods provide saliency maps to highlight important input features of a specific eye gaze sequence. However, to date, its localization analysis has been lacking a quantitative approach across entire datasets. In this work, we employ established gaze event detection algorithms for fixations and saccades and quantitatively evaluate the impact of these events by determining their concept influence. Input features that belong to saccades are shown to be substantially more important than features that belong to fixations. By dissecting saccade events into sub-events, we are able to show that gaze samples that are close to the saccadic peak velocity are most influential. We further investigate the effect of event properties like saccadic amplitude or fixational dispersion on the resulting concept influence.

Related papers

• Explaining Human Activity Recognition with SHAP: Validating Insights with Perturbation and Quantitative Measures [0.1398098625978622]
  This study uses SHapley Additive exPlanations to explain the decision-making process of Graph Convolution Networks (GCNs) We employ SHAP to explain two real-world datasets: one for cerebral palsy (CP) classification and the widely used NTU RGB+D 60 action recognition dataset. Results on both datasets show that body key points identified as important through SHAP have the largest influence on the accuracy, specificity, and sensitivity metrics.
  arXiv  Detail & Related papers  (2024-11-06T07:28:57Z)
• Localized Gaussians as Self-Attention Weights for Point Clouds Correspondence [92.07601770031236]
  We investigate semantically meaningful patterns in the attention heads of an encoder-only Transformer architecture. We find that fixing the attention weights not only accelerates the training process but also enhances the stability of the optimization.
  arXiv  Detail & Related papers  (2024-09-20T07:41:47Z)
• Visual Context-Aware Person Fall Detection [52.49277799455569]
  We present a segmentation pipeline to semi-automatically separate individuals and objects in images. Background objects such as beds, chairs, or wheelchairs can challenge fall detection systems, leading to false positive alarms. We demonstrate that object-specific contextual transformations during training effectively mitigate this challenge.
  arXiv  Detail & Related papers  (2024-04-11T19:06:36Z)
• What Matters When Repurposing Diffusion Models for General Dense Perception Tasks? [49.84679952948808]
  Recent works show promising results by simply fine-tuning T2I diffusion models for dense perception tasks. We conduct a thorough investigation into critical factors that affect transfer efficiency and performance when using diffusion priors. Our work culminates in the development of GenPercept, an effective deterministic one-step fine-tuning paradigm tailed for dense visual perception tasks.
  arXiv  Detail & Related papers  (2024-03-10T04:23:24Z)
• Deep graph kernel point processes [19.382241594513374]
  This paper presents a novel point process model for discrete event data over graphs, where the event interaction occurs within a latent graph structure. The key idea is to represent the influence kernel by Graph Neural Networks (GNN) to capture the underlying graph structure. Compared with prior works focusing on directly modeling the conditional intensity function using neural networks, our kernel presentation herds the repeated event influence patterns more effectively.
  arXiv  Detail & Related papers  (2023-06-20T06:15:19Z)
• LatentGaze: Cross-Domain Gaze Estimation through Gaze-Aware Analytic Latent Code Manipulation [0.0]
  We propose a gaze-aware analytic manipulation method, based on a data-driven approach with generative adversarial network inversion's disentanglement characteristics. By utilizing GAN-based encoder-generator process, we shift the input image from the target domain to the source domain image, which a gaze estimator is sufficiently aware.
  arXiv  Detail & Related papers  (2022-09-21T08:05:53Z)
• Learning Neural Causal Models with Active Interventions [83.44636110899742]
  We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process. Our method significantly reduces the required number of interactions compared with random intervention targeting. We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
  arXiv  Detail & Related papers  (2021-09-06T13:10:37Z)
• What Image Features Boost Housing Market Predictions? [81.32205133298254]
  We propose a set of techniques for the extraction of visual features for efficient numerical inclusion in predictive algorithms. We discuss techniques such as Shannon's entropy, calculating the center of gravity, employing image segmentation, and using Convolutional Neural Networks. The set of 40 image features selected here carries a significant amount of predictive power and outperforms some of the strongest metadata predictors.
  arXiv  Detail & Related papers  (2021-07-15T06:32:10Z)
• Wave Propagation of Visual Stimuli in Focus of Attention [77.4747032928547]
  Fast reactions to changes in the surrounding visual environment require efficient attention mechanisms to reallocate computational resources to most relevant locations in the visual field. We present a biologically-plausible model of focus of attention that exhibits effectiveness and efficiency exhibited by foveated animals.
  arXiv  Detail & Related papers  (2020-06-19T09:33:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.