A Probabilistic Time-Evolving Approach to Scanpath Prediction
- URL: http://arxiv.org/abs/2204.09404v1
- Date: Wed, 20 Apr 2022 11:50:29 GMT
- Title: A Probabilistic Time-Evolving Approach to Scanpath Prediction
- Authors: Daniel Martin, Diego Gutierrez, Belen Masia
- Abstract summary: We present a probabilistic time-evolving approach to scanpath prediction, based on Bayesian deep learning.
Our model yields results that outperform those of current state-of-the-art approaches, and are almost on par with the human baseline.
- Score: 8.669748138523758
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Human visual attention is a complex phenomenon that has been studied for
decades. Within it, the particular problem of scanpath prediction poses a
challenge, particularly due to the inter- and intra-observer variability, among
other reasons. Besides, most existing approaches to scanpath prediction have
focused on optimizing the prediction of a gaze point given the previous ones.
In this work, we present a probabilistic time-evolving approach to scanpath
prediction, based on Bayesian deep learning. We optimize our model using a
novel spatio-temporal loss function based on a combination of Kullback-Leibler
divergence and dynamic time warping, jointly considering the spatial and
temporal dimensions of scanpaths. Our scanpath prediction framework yields
results that outperform those of current state-of-the-art approaches, and are
almost on par with the human baseline, suggesting that our model is able to
generate scanpaths whose behavior closely resembles those of the real ones.
Related papers
- GazeXplain: Learning to Predict Natural Language Explanations of Visual Scanpaths [20.384132849805003]
We introduce GazeXplain, a novel study of visual scanpath prediction and explanation.
This involves annotating natural-language explanations for fixations across eye-tracking datasets.
Experiments on diverse eye-tracking datasets demonstrate the effectiveness of GazeXplain in both scanpath prediction and explanation.
arXiv Detail & Related papers (2024-08-05T19:11:46Z) - Improving Trajectory Prediction in Dynamic Multi-Agent Environment by
Dropping Waypoints [9.385936248154987]
Motion prediction systems must learn spatial and temporal information from the past to forecast the future trajectories of the agent.
We propose Temporal Waypoint Dropping (TWD) that explicitly incorporates temporal dependencies during the training of a trajectory prediction model.
We evaluate our proposed approach on three datasets: NBA Sports VU, ETH-UCY, and TrajNet++.
arXiv Detail & Related papers (2023-09-29T15:48:35Z) - Scanpath Prediction in Panoramic Videos via Expected Code Length
Minimization [27.06179638588126]
We present a new criterion for scanpath prediction based on principles from lossy data compression.
This criterion suggests minimizing the expected code length of quantized scanpaths in a training set.
We also introduce a proportional-integral-derivative (PID) controller-based sampler to generate realistic human-like scanpaths.
arXiv Detail & Related papers (2023-05-04T04:10:47Z) - Uncovering the Missing Pattern: Unified Framework Towards Trajectory
Imputation and Prediction [60.60223171143206]
Trajectory prediction is a crucial undertaking in understanding entity movement or human behavior from observed sequences.
Current methods often assume that the observed sequences are complete while ignoring the potential for missing values.
This paper presents a unified framework, the Graph-based Conditional Variational Recurrent Neural Network (GC-VRNN), which can perform trajectory imputation and prediction simultaneously.
arXiv Detail & Related papers (2023-03-28T14:27:27Z) - Sinkhorn-Flow: Predicting Probability Mass Flow in Dynamical Systems
Using Optimal Transport [89.61692654941106]
We propose a new approach to predicting such mass flow over time using optimal transport.
We apply our approach to the task of predicting how communities will evolve over time in social network settings.
arXiv Detail & Related papers (2023-03-14T07:25:44Z) - TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction [64.63645677568384]
We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals.
Our approach locally modulates the saliency predictions by combining the learned temporal maps.
Our code will be publicly available on GitHub.
arXiv Detail & Related papers (2023-01-05T22:10:16Z) - An Inter-observer consistent deep adversarial training for visual
scanpath prediction [66.46953851227454]
We propose an inter-observer consistent adversarial training approach for scanpath prediction through a lightweight deep neural network.
We show the competitiveness of our approach in regard to state-of-the-art methods.
arXiv Detail & Related papers (2022-11-14T13:22:29Z) - Probabilistic Human Motion Prediction via A Bayesian Neural Network [71.16277790708529]
We propose a probabilistic model for human motion prediction in this paper.
Our model could generate several future motions when given an observed motion sequence.
We extensively validate our approach on a large scale benchmark dataset Human3.6m.
arXiv Detail & Related papers (2021-07-14T09:05:33Z) - Temporally-Continuous Probabilistic Prediction using Polynomial
Trajectory Parameterization [12.896275507449936]
A commonly-used representation for motion prediction of actors is a sequence of waypoints for each actor at discrete future time-points.
This approach is simple and flexible, but it can exhibit unrealistic higher-order derivatives and approximation errors at intermediate time steps.
We propose a simple and general representation for temporally continuous trajectory prediction that is based on trajectory parameterization.
arXiv Detail & Related papers (2020-11-01T01:51:44Z) - Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
We propose an integrated solution based on the deep neural networks for temporal sets prediction.
A unique perspective is to learn element relationship by constructing set-level co-occurrence graph.
We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
arXiv Detail & Related papers (2020-06-20T03:29:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.