TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction

• URL: http://arxiv.org/abs/2301.02315v2
• Date: Tue, 10 Sep 2024 10:57:14 GMT
• Title: TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction
• Authors: Bahar Aydemir, Ludo Hoffstetter, Tong Zhang, Mathieu Salzmann, Sabine Süsstrunk,
• Abstract summary: We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our code will be publicly available on GitHub.
• Score: 64.63645677568384
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity. However, none of these models consider the temporal nature of gaze shifts during image observation. We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals by exploiting human temporal attention patterns. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our experiments show that our method outperforms the state-of-the-art models, including a multi-duration saliency model, on the SALICON benchmark. Our code will be publicly available on GitHub.

Related papers

• TPP-Gaze: Modelling Gaze Dynamics in Space and Time with Neural Temporal Point Processes [63.95928298690001]
  We present TPP-Gaze, a novel and principled approach to model scanpath dynamics based on Neural Temporal Point Process (TPP) Our results show the overall superior performance of the proposed model compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-10-30T19:22:38Z)
• Precipitation nowcasting with generative diffusion models [0.0]
  We study the efficacy of diffusion models in handling the task of precipitation nowcasting. Our work is conducted in comparison to the performance of well-established U-Net models.
  arXiv  Detail & Related papers  (2023-08-13T09:51:16Z)
• OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning [67.07363529640784]
  We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather. We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
  arXiv  Detail & Related papers  (2023-06-20T03:02:14Z)
• Mixed moving average field guided learning for spatio-temporal data [0.0]
  We define a novel Bayesian-temporal embedding and a theory-guided machine learning approach to make ensemble forecasts. We use Lipschitz predictors to determine fixed-time and any-time PAC in the batch learning setting. We then test the performance of our learning methodology by using linear predictors and data sets simulated from a dependence- Ornstein-Uhlenbeck process.
  arXiv  Detail & Related papers  (2023-01-02T16:11:05Z)
• PRISM: Probabilistic Real-Time Inference in Spatial World Models [52.878769723544615]
  PRISM is a method for real-time filtering in a probabilistic generative model of agent motion and visual perception. The proposed solution runs at 10Hz real-time and is similarly accurate to state-of-the-art SLAM in small to medium-sized indoor environments.
  arXiv  Detail & Related papers  (2022-12-06T13:59:06Z)
• Temporal Relevance Analysis for Video Action Models [70.39411261685963]
  We first propose a new approach to quantify the temporal relationships between frames captured by CNN-based action models. We then conduct comprehensive experiments and in-depth analysis to provide a better understanding of how temporal modeling is affected.
  arXiv  Detail & Related papers  (2022-04-25T19:06:48Z)
• RecSal : Deep Recursive Supervision for Visual Saliency Prediction [2.223733768286313]
  Saliency prediction datasets can be used to create more information for each stimulus than just a final aggregate saliency map. We show that our best method outperforms previous state-of-the-art methods with 50-80% fewer parameters.
  arXiv  Detail & Related papers  (2020-08-31T17:08:34Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.