Do vision-language models search like humans? Reasoning tokens as a reaction-time analog in classic visual-search paradigms

• URL: http://arxiv.org/abs/2606.25066v1
• Date: Tue, 23 Jun 2026 18:19:16 GMT
• Title: Do vision-language models search like humans? Reasoning tokens as a reaction-time analog in classic visual-search paradigms
• Abstract summary: I ask whether vision-language models (VLMs) exhibit the same behavioral signatures.<n>I adapt four classic paradigms: feature versus conjunction search, spatial-configuration (T-vs-L) search, enumeration, and the tilted/vertical search asymmetry.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual search has been one of the most productive paradigms in the study of visual attention: the way reaction time scales with the number of items distinguishes parallel, "pop-out" search from serial, attention-demanding search. I ask whether vision-language models (VLMs) exhibit the same behavioral signatures. I adapt four classic paradigms: feature versus conjunction search, spatial-configuration (T-vs-L) search, enumeration, and the tilted/vertical search asymmetry; and present them to current frontier and mid-tier models. Because a single model call has no reaction time, I use the number of reasoning ("thinking") tokens a model spends per trial as a within-model analog of search effort, and I compare against a large public human benchmark (Wolfe et al., 2010). The models reproduce several human signatures: feature search costs flat effort while conjunction effort climbs with set size; frontier models hold accuracy where mid-tier models collapse to chance; and a resolution control shows the conjunction cost is genuine search rather than difficulty resolving small shapes. They also diverge from humans in informative ways. The target-present effort slope exceeds the target-absent slope, reversing the human ordering; enumeration remains accurate where humans would lose count; and a reasoning model with adaptive deliberation declines to deliberate on detection tasks altogether, so that a single search expresses itself as an effort gradient in one model and as an accuracy cliff in another. I argue that psychophysical paradigms, applied behaviorally, are a sharp and inexpensive probe of machine visual cognition, and that the points of divergence are as informative as the points of agreement.

Related papers

• Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans [50.71222402701153]
  We compare three general-purpose large language models (MLLMs) with human eye-movement scanpaths on goal-directed search (COCO-Search18)<n>All three share one signature: low-entropy, large-amplitude, self-consistent scanpaths that agree with themselves far more closely than two humans agree with each other.<n>The gap sits on a process axis that answer-alignment and saliency metrics do not measure. Because they miss it, such metrics cannot certify human-like vision, and zero-shot models suit outcome and spatial questions but not temporal, process-level
  arXiv  Detail & Related papers  (2026-08-17T12:54:39Z)
• Modeling Human Gaze Behavior with Diffusion Models for Unified Scanpath Prediction [66.71402249062777]
  We present ScanDiff, a novel architecture that combines diffusion models with Vision Transformers to generate diverse and realistic scanpaths.<n>Our method explicitly models scanpath variability by leveraging the nature of diffusion models, producing a wide range of plausible gaze trajectories.<n>Experiments on benchmark datasets show that ScanDiff surpasses state-of-the-art methods in both free-viewing and task-driven scenarios.
  arXiv  Detail & Related papers  (2025-07-30T18:36:09Z)
• Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models [51.900488744931785]
  We introduce the Visual Graph Arena (VGA) to evaluate and improve AI systems' capacity for visual abstraction.<n>Humans achieve near-perfect accuracy across tasks, while models totally failed on isomorphism detection and showed limited success in path/cycle tasks.<n>By isolating the challenge of representation-invariant reasoning, the VGA provides a framework to drive progress toward human-like conceptualization in AI visual models.
  arXiv  Detail & Related papers  (2025-06-06T17:06:25Z)
• Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models [58.64449765678416]
  We introduce landscape of thoughts (LoT) to inspect the reasoning trajectories with certain reasoning methods on any multi-choice dataset.<n>LoT distinguishes between strong and weak models, correct and incorrect answers, as well as different reasoning tasks.<n>We showcase this advantage by adapting LoT to a lightweight verifier that evaluates the correctness of trajectories.
  arXiv  Detail & Related papers  (2025-03-28T06:09:51Z)
• Evaluating Multiview Object Consistency in Humans and Image Models [68.36073530804296]
  We leverage an experimental design from the cognitive sciences which requires zero-shot visual inferences about object shape. We collect 35K trials of behavioral data from over 500 participants. We then evaluate the performance of common vision models.
  arXiv  Detail & Related papers  (2024-09-09T17:59:13Z)
• Longer Fixations, More Computation: Gaze-Guided Recurrent Neural Networks [12.57650361978445]
  Humans read texts at a varying pace, while machine learning models treat each token in the same way. In this paper, we convert this intuition into a set of novel models with fixation-guided parallel RNNs or layers. We find that, interestingly, the fixation duration predicted by neural networks bears some resemblance to humans' fixation.
  arXiv  Detail & Related papers  (2023-10-31T21:32:11Z)
• Computing a human-like reaction time metric from stable recurrent vision models [11.87006916768365]
  We sketch a general-purpose methodology to construct computational accounts of reaction times from a stimulus-computable, task-optimized model. We demonstrate that our metric aligns with patterns of human reaction times for stimulus manipulations across four disparate visual decision-making tasks. This work paves the way for exploring the temporal alignment of model and human visual strategies in the context of various other cognitive tasks.
  arXiv  Detail & Related papers  (2023-06-20T14:56:02Z)
• Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango [11.344587937052697]
  This work initiates the preliminary steps towards a deeper understanding of reasoning mechanisms in large language models. Our work centers around querying the model while controlling for all but one of the components in a prompt: symbols, patterns, and text. We posit that text imbues patterns with commonsense knowledge and meaning.
  arXiv  Detail & Related papers  (2022-09-16T02:54:00Z)
• Perceptual Score: What Data Modalities Does Your Model Perceive? [73.75255606437808]
  We introduce the perceptual score, a metric that assesses the degree to which a model relies on the different subsets of the input features. We find that recent, more accurate multi-modal models for visual question-answering tend to perceive the visual data less than their predecessors. Using the perceptual score also helps to analyze model biases by decomposing the score into data subset contributions.
  arXiv  Detail & Related papers  (2021-10-27T12:19:56Z)
• AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses [66.49753193098356]
  We investigate the reason behind the surprising adversarial brittleness of scoring models. Our results indicate that autoscoring models, despite getting trained as "end-to-end" models, behave like bag-of-words models. We propose detection-based protection models that can detect oversensitivity and overstability causing samples with high accuracies.
  arXiv  Detail & Related papers  (2021-09-24T03:49:38Z)
• Visual Search Asymmetry: Deep Nets and Humans Share Similar Inherent Biases [7.209733010493132]
  An intriguing property of some classical search tasks is an asymmetry such that finding a target A among distractors B can be easier than finding B among A. We propose a computational model that takes a target and a search image as inputs and produces a sequence of eye movements until the target is found. We compared the model against human behavior in six paradigmatic search tasks that show asymmetry in humans.
  arXiv  Detail & Related papers  (2021-06-05T19:46:42Z)
• Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning [44.774961463015245]
  We propose the first inverse reinforcement learning model to learn the internal reward function and policy used by humans during visual search. To train and evaluate our IRL model we created COCO-Search18, which is now the largest dataset of high-quality search fixations in existence.
  arXiv  Detail & Related papers  (2020-05-28T21:46:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.