Do Vision-Language-Models show human-like logical problem-solving capability in point and click puzzle games?

• URL: http://arxiv.org/abs/2605.11223v2
• Date: Sun, 17 May 2026 19:07:54 GMT
• Title: Do Vision-Language-Models show human-like logical problem-solving capability in point and click puzzle games?
• Abstract summary: This paper introduces Vision-Language Against The Incredible Machine (VLATIM), a benchmark designed to evaluate human-like logical problem-solving capabilities.<n>Unlike existing benchmarks, VLATIM specifically targets the critical gap between high-level logical reasoning and continuous action spaces.<n>Our results reveal a significant disparity between reasoning and execution.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language(-Action) Models (VLMs) are increasingly applied to interactive environments, yet existing benchmarks often overlook the complex physical reasoning required for point-and-click puzzle games. This paper introduces Vision-Language Against The Incredible Machine (VLATIM), a benchmark designed to evaluate human-like logical problem-solving capabilities within the classic physics puzzle game The Incredible Machine 2 (TIM). Unlike existing benchmarks, VLATIM specifically targets the critical gap between high-level logical reasoning and continuous action spaces requiring precise mouse interactions. This benchmark is structured into five progressive parts, assessing capabilities that range from basic visual grounding and domain understanding to multi-step manipulation and full puzzle solving. Our results reveal a significant disparity between reasoning and execution. While large proprietary models demonstrate superior planning abilities, they struggle with precise visual grounding. Consequently, they do not yet show human-like problem-solving capabilities.

Related papers

• PuzzleMate: Benchmarking MLLMs for Egocentric Puzzle Assistance [21.60284785247072]
  We use PuzzleMate to evaluate how well state-of-the-art MLLMs perceive the current puzzle state and generate actionable next-step instructions.<n>Our findings reveal a substantial performance gap in current models like GPT-5.2 and Gemini-2.5-Pro.
  arXiv  Detail & Related papers  (2026-09-13T12:29:03Z)
• From Perception to Action: An Interactive Benchmark for Vision Reasoning [51.11355591375073]
  Causal Hierarchy of Actions and Interactions (CHAIN) benchmark designed to evaluate whether models can understand, plan, and execute structured action sequences grounded in physical constraints.<n> CHAIN shifts evaluation from passive perception to active problem solving, spanning tasks such as interlocking mechanical puzzles and 3D stacking and packing.<n>Our results show that top-performing models still struggle to internalize physical structure and causal constraints, often failing to produce reliable long-horizon plans and cannot robustly translate perceived structure into effective actions.
  arXiv  Detail & Related papers  (2026-02-24T15:33:02Z)
• SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models [60.088066516175026]
  We introduce a benchmark designed to evaluate the spatial logical reasoning capabilities of Vision-Language Models (VLMs)<n>We conduct extensive experiments on 41 mainstream VLMs, and the results show that even the most advanced models still struggle with spatial logical reasoning.<n>We propose a method called recursive scene graph assisted reasoning, which leverages visual foundation models to progressively decompose complex scenes into task-relevant scene graphs.
  arXiv  Detail & Related papers  (2026-02-24T13:38:37Z)
• Eye-Q: A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning [1.6234264741872295]
  Vision-Language Models (VLMs) have achieved strong performance on standard vision-language benchmarks.<n>We propose visual word puzzles as a challenging alternative, as they require discovering implicit visual cues, generating and revising hypotheses, and mapping evidence to non-literal concepts.<n>We introduce Eye-Q, a multilingual benchmark designed to assess this form of complex visual understanding.
  arXiv  Detail & Related papers  (2026-01-06T20:27:29Z)
• BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception [67.89135437537179]
  We introduce BLINK-Twice, a vision-centric reasoning benchmark grounded in challenging perceptual tasks.<n>Instead of relying on external knowledge, our tasks require models to reason from visual content alone.<n>Compared to prior perception benchmarks, it moves beyond shallow perception and requires fine-grained observation and analytical reasoning.
  arXiv  Detail & Related papers  (2025-10-10T13:14:13Z)
• Game-Time: Evaluating Temporal Dynamics in Spoken Language Models [93.844257719952]
  We introduce the Game-Time Benchmark framework to assess temporal capabilities.<n>Our evaluation of diverse SLM models reveals a clear performance disparity.<n>The GameTime Benchmark provides a foundation for guiding future research toward more temporally-aware conversational AI.
  arXiv  Detail & Related papers  (2025-09-30T15:23:39Z)
• GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games [7.594173359523366]
  We introduce GVGAI-LLM, a video game benchmark for evaluating the reasoning and problem-solving capabilities of large language models (LLMs)<n>Built on the General Video Game AI framework, it features a diverse collection of arcade-style games designed to test a model's ability to handle tasks that differ from most existing LLM benchmarks.
  arXiv  Detail & Related papers  (2025-08-11T22:17:07Z)
• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations [61.235500325327585]
  Existing AI benchmarks primarily assess verbal reasoning, neglecting the complexities of non-verbal, multi-step visual simulation.<n>We introduce STARE, a benchmark designed to rigorously evaluate multimodal large language models on tasks better solved through visual simulation.<n>Our evaluations show that models excel at reasoning over simpler 2D transformations, but perform close to random chance on more complex tasks.
  arXiv  Detail & Related papers  (2025-06-05T05:09:46Z)
• VisualPuzzles: Decoupling Multimodal Reasoning Evaluation from Domain Knowledge [45.20691825097646]
  We introduce VisualPuzzles, a benchmark that targets visual reasoning.<n>VisualPuzzles consists of diverse questions spanning five categories: algorithmic, analogical, deductive, inductive, and spatial reasoning.
  arXiv  Detail & Related papers  (2025-04-14T15:50:39Z)
• V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models [84.27290155010533]
  We introduce Vision-centric Multiple Abilities Game Evaluation (V-MAGE), a novel game-based evaluation framework.<n>V-MAGE features five distinct video games comprising over 30 carefully constructed evaluation scenarios.<n>We show V-MAGE provides actionable insights for improving the visual and reasoning capabilities of MLLMs in dynamic, interactive settings.
  arXiv  Detail & Related papers  (2025-04-08T15:43:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.