TDSal: Task-Based Top-Down Saliency Prediction Model

• URL: http://arxiv.org/abs/2607.09827v1
• Date: Fri, 10 Jul 2026 11:47:55 GMT
• Title: TDSal: Task-Based Top-Down Saliency Prediction Model
• Abstract summary: We propose a model that conditions visual attention on natural-language task descriptions.<n>The model produces task-dependent saliency maps that reflect how attention shifts under different viewing intents.<n>We show that incorporating explicit task semantics enables more faithful modeling of goal-directed visual attention.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual saliency aims to predict the regions of an image most likely to attract human visual attention. While most saliency models assume free-viewing conditions, human attention is often shaped by explicit task goals. In this work, we address task-driven saliency prediction by proposing a model that conditions visual attention on natural-language task descriptions. The model produces task-dependent saliency maps that reflect how attention shifts under different viewing intents. Through quantitative and qualitative analysis, we show that incorporating explicit task semantics enables more faithful modeling of goal-directed visual attention.

Related papers

• Emergent Goal-Directed Attention in Large Vision-Language Models [3.2171716419695335]
  General-purpose vision-language models (VLMs) can generate human-aligned, goal-directed spatial priorities without gaze-specific training.<n>These findings show that general-purpose VLMs can generate human-aligned, goal-directed spatial priorities without gaze-specific training.
  arXiv  Detail & Related papers  (2026-08-31T15:00:02Z)
• Human Scanpath Prediction in Target-Present Visual Search with Semantic-Foveal Bayesian Attention [49.99728312519117]
  SemBA-FAST is a top-down framework designed for predicting human visual attention in target-present visual search.<n>We evaluate SemBA-FAST on the COCO-Search18 benchmark dataset, comparing its performance against other scanpath prediction models.<n>These findings provide valuable insights into the capabilities of semantic-foveal probabilistic frameworks for human-like attention modelling.
  arXiv  Detail & Related papers  (2025-07-24T15:19:23Z)
• Semantic-Based Active Perception for Humanoid Visual Tasks with Foveal Sensors [49.99728312519117]
  The aim of this work is to establish how accurately a recent semantic-based active perception model is able to complete visual tasks that are regularly performed by humans. This model exploits the ability of current object detectors to localize and classify a large number of object classes and to update a semantic description of a scene across multiple fixations. In the task of scene exploration, the semantic-based method demonstrates superior performance compared to the traditional saliency-based model.
  arXiv  Detail & Related papers  (2024-04-16T18:15:57Z)
• GazeFusion: Saliency-Guided Image Generation [50.37783903347613]
  Diffusion models offer unprecedented image generation power given just a text prompt.<n>We present a saliency-guided framework to incorporate the data priors of human visual attention mechanisms into the generation process.
  arXiv  Detail & Related papers  (2024-03-16T21:01:35Z)
• Top-Down Visual Attention from Analysis by Synthesis [87.47527557366593]
  We consider top-down attention from a classic Analysis-by-Synthesis (AbS) perspective of vision. We propose Analysis-by-Synthesis Vision Transformer (AbSViT), which is a top-down modulated ViT model that variationally approximates AbS, and controllable achieves top-down attention.
  arXiv  Detail & Related papers  (2023-03-23T05:17:05Z)
• Task Bias in Vision-Language Models [18.025004053980545]
  We explore the CLIP model and show that its visual representation is often strongly biased towards solving some tasks more than others. To resolve this task bias, we show how to learn a visual prompt that guides the representation towards features relevant to their task of interest.
  arXiv  Detail & Related papers  (2022-12-08T17:10:31Z)
• Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration [47.01485765231528]
  Active visual exploration aims to assist an agent with a limited field of view to understand its environment based on partial observations. We propose the Glimpse-Attend-and-Explore model which employs self-attention to guide the visual exploration instead of task-specific uncertainty maps. Our model provides encouraging results while being less dependent on dataset bias in driving the exploration.
  arXiv  Detail & Related papers  (2021-08-26T11:41:03Z)
• Goal-Aware Prediction: Learning to Model What Matters [105.43098326577434]
  One of the fundamental challenges in using a learned forward dynamics model is the mismatch between the objective of the learned model and that of the downstream planner or policy. We propose to direct prediction towards task relevant information, enabling the model to be aware of the current task and encouraging it to only model relevant quantities of the state space. We find that our method more effectively models the relevant parts of the scene conditioned on the goal, and as a result outperforms standard task-agnostic dynamics models and model-free reinforcement learning.
  arXiv  Detail & Related papers  (2020-07-14T16:42:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.