TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction
- URL: http://arxiv.org/abs/2301.02315v1
- Date: Thu, 5 Jan 2023 22:10:16 GMT
- Title: TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction
- Authors: Bahar Aydemir, Ludo Hoffstetter, Tong Zhang, Mathieu Salzmann, Sabine
S\"usstrunk
- Abstract summary: We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals.
Our approach locally modulates the saliency predictions by combining the learned temporal maps.
Our code will be publicly available on GitHub.
- Score: 56.22339016797785
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep saliency prediction algorithms complement the object recognition
features, they typically rely on additional information, such as scene context,
semantic relationships, gaze direction, and object dissimilarity. However, none
of these models consider the temporal nature of gaze shifts during image
observation. We introduce a novel saliency prediction model that learns to
output saliency maps in sequential time intervals by exploiting human temporal
attention patterns. Our approach locally modulates the saliency predictions by
combining the learned temporal maps. Our experiments show that our method
outperforms the state-of-the-art models, including a multi-duration saliency
model, on the SALICON benchmark. Our code will be publicly available on GitHub.
Related papers
- Temporal Smoothness Regularisers for Neural Link Predictors [9.80506802512948]
We show that a simple method like TNTComplEx can produce significantly more accurate results than state-of-the-art methods.
We also evaluate the impact of a wide range of temporal smoothing regularisers on two state-of-the-art temporal link prediction models.
arXiv Detail & Related papers (2023-09-16T16:52:49Z) - Precipitation nowcasting with generative diffusion models [0.0]
We study the efficacy of diffusion models in handling the task of precipitation nowcasting.
Our work is conducted in comparison to the performance of well-established U-Net models.
arXiv Detail & Related papers (2023-08-13T09:51:16Z) - OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive
Learning [67.07363529640784]
We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models.
We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather.
We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
arXiv Detail & Related papers (2023-06-20T03:02:14Z) - Mixed moving average field guided learning for spatio-temporal data [0.0]
We use a theory-guided machine learning approach to make ensemble forecasts.
We highlight its potential application to data with spatialtemporal and temporal short and long-range sets.
We then test the performance of our learning methodology by using linear predictors and data sets simulated from a dependence- Ornstein-Uhlenbeck process.
arXiv Detail & Related papers (2023-01-02T16:11:05Z) - PRISM: Probabilistic Real-Time Inference in Spatial World Models [52.878769723544615]
PRISM is a method for real-time filtering in a probabilistic generative model of agent motion and visual perception.
The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments.
arXiv Detail & Related papers (2022-12-06T13:59:06Z) - Temporal Relevance Analysis for Video Action Models [70.39411261685963]
We first propose a new approach to quantify the temporal relationships between frames captured by CNN-based action models.
We then conduct comprehensive experiments and in-depth analysis to provide a better understanding of how temporal modeling is affected.
arXiv Detail & Related papers (2022-04-25T19:06:48Z) - Visualising Deep Network's Time-Series Representations [93.73198973454944]
Despite the popularisation of machine learning models, more often than not they still operate as black boxes with no insight into what is happening inside the model.
In this paper, a method that addresses that issue is proposed, with a focus on visualising multi-dimensional time-series data.
Experiments on a high-frequency stock market dataset show that the method provides fast and discernible visualisations.
arXiv Detail & Related papers (2021-03-12T09:53:34Z) - RecSal : Deep Recursive Supervision for Visual Saliency Prediction [2.223733768286313]
Saliency prediction datasets can be used to create more information for each stimulus than just a final aggregate saliency map.
We show that our best method outperforms previous state-of-the-art methods with 50-80% fewer parameters.
arXiv Detail & Related papers (2020-08-31T17:08:34Z) - Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
We propose an integrated solution based on the deep neural networks for temporal sets prediction.
A unique perspective is to learn element relationship by constructing set-level co-occurrence graph.
We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
arXiv Detail & Related papers (2020-06-20T03:29:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.