TraversalBench: Challenging Paths to Follow for Vision Language Models

• URL: http://arxiv.org/abs/2604.10999v1
• Date: Mon, 13 Apr 2026 04:58:52 GMT
• Title: TraversalBench: Challenging Paths to Follow for Vision Language Models
• Abstract summary: Vision-language models (VLMs) perform strongly on multimodal benchmarks, but the ability to follow complex visual paths is under-tested.<n>We introduce TraversalBench, a benchmark for exact visual path traversal.<n>We find that self-intersections are the dominant source of difficulty.
• Score: 11.032462608031922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) perform strongly on many multimodal benchmarks. However, the ability to follow complex visual paths -- a task that human observers typically find straightforward -- remains under-tested. We introduce TraversalBench, a controlled benchmark for exact visual path traversal. Each instance contains a single continuous polyline, a unique start marker, and markers placed at path vertices; the task is to recover the exact ordered sequence encountered when traversing the path from start to finish. The benchmark explicitly balances key path-structural factors including self-intersection count, tortuosity, vertex count, and nearby confounding lines, while minimizing reliance on OCR, world knowledge, and open-ended planning. We find that self-intersections are the dominant source of difficulty. A first-crossing analysis shows that errors are sharply localized: performance is relatively stable immediately before the first crossing, then drops steeply when the model must resolve the correct continuation. By contrast, nearby confounding lines produce a weaker persistent degradation that compounds with repeated exposure. These analyses make TraversalBench a useful diagnostic for identifying whether models suffer from human-like failures or other breakdowns in sustained visual processing. An auxiliary reading-order benchmark further reveals a consistent preference for layouts compatible with left-to-right serialization, while not explaining away the main effects of path complexity. Together, these results position TraversalBench as a controlled diagnostic of path-faithful visual reasoning and as a useful testbed for studying multimodal spatial reasoning under ambiguity, clutter, and distractor structure. More broadly, we position TraversalBench as a contribution to the still-limited area of sustained visual grounding benchmarks for VLMs.

Related papers

• Is Visual Prompting All You Need? Studying VLM Spatial Reasoning under Progressive Visual Scaffolds [12.841957617988148]
  We study how the visual presentation of a task shapes model performance and failure modes when the underlying reasoning problem is unchanged.<n>We introduce lightweight input-side scaffolds that preserve the visual modality while making spatial structure more accessible.<n>We find that visual presentation is a central factor that determines whether VLM benchmarks measure grounded perception, downstream reasoning, or a mixture of both.
  arXiv  Detail & Related papers  (2026-08-21T14:40:26Z)
• TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference [47.62941623025392]
  We present TELLER, a non-intrusive and Log-aware LLM inference Root-cause analysis framework.<n>TELLER collects NVTX/CUPTI traces and service logs without modifying model binaries.<n>It reconstructs per-request call-chain trees and aligns log lines with corresponding execution steps.<n>On top of these representations, TELLER combines numeric candidate localization with a multimodal root-cause model.
  arXiv  Detail & Related papers  (2026-08-03T09:39:11Z)
• VLMs Trace Without Tracking: Diagnosing Failures in Visual Path Following [10.22092497929711]
  Vision-language models (VLMs) achieve strong performance on multimodal benchmarks, but may still lack robust control over basic visual operations.<n>We study textitline tracing, where a model must follow a selected visual path through successive local continuations.<n>We design controlled tracing tasks that introduce nearby competitors while reducing semantic and topological ambiguity.
  arXiv  Detail & Related papers  (2026-05-15T06:48:53Z)
• The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space [29.35702264173152]
  We identify a pervasive vulnerability, the textbfCartesian Shortcut: visual reasoning benchmarks prevalently build on orthogonal grid-based layouts.<n>Models systematically exploit this property, heavily leveraging text-based deductive reasoning to assist visual problem-solving.<n>We introduce textbfPolaris-Bench, which re-formulates 53 visual reasoning tasks in Polar coordinate space with paired Cartesian counterparts as reference.
  arXiv  Detail & Related papers  (2026-05-11T02:16:48Z)
• Evading Visual Aphasia: Contrastive Adaptive Semantic Token Pruning for Vision-Language Models [52.78477729846771]
  We introduce COAST (COntrastive Adaptive Semantic Token Pruning), a training-free pruning framework that casts compression as adaptive semantic routing.<n> COAST consistently outperforms strong pruning baselines across token budgets and generalizes across multiple LVLM families.
  arXiv  Detail & Related papers  (2026-05-10T09:07:04Z)
• An Embarrassingly Simple Graph Heuristic Reveals Shortcut-Solvable Benchmarks for Sequential Recommendation [50.09718257952108]
  Sequentialsolv has increasingly shifted toward generative recommenders that combine sequential patterns with semantic item information.<n>These methods are often evaluated on a small set of widely used benchmarks, raising a key question: do these benchmarks actually require the advanced modeling capabilities that modern generative recommenders claim to provide?<n>We conduct a benchmark audit with an intentionally simple graph. Starting from only the last one or two items, it retrieves candidates from a few-hop item-transition graph and ranks them by item-feature similarity.
  arXiv  Detail & Related papers  (2026-05-08T02:00:11Z)
• Guideline2Graph: Profile-Aware Multimodal Parsing for Executable Clinical Decision Graphs [3.459382629188014]
  We present a decomposition-first pipeline that converts full-guideline evidence into an executable clinical decision graph.<n>We evaluate on an adjudicated prostate-guideline benchmark with matched inputs and the same underlying VLM backbone.
  arXiv  Detail & Related papers  (2026-04-02T19:21:31Z)
• CRIT: Graph-Based Automatic Data Synthesis to Enhance Cross-Modal Multi-Hop Reasoning [28.037871701844654]
  We introduce CRIT, a new dataset and benchmark built with a graph-based automatic pipeline for generating complex cross-modal reasoning tasks.<n>CRIT consists of diverse domains ranging from natural images, videos, and text-rich sources, and includes a manually verified test set for reliable evaluation.<n>Models trained on CRIT show significant gains in cross-modal multi-hop reasoning, including strong improvements on SPIQA and other standard multimodal benchmarks.
  arXiv  Detail & Related papers  (2026-04-02T05:23:18Z)
• Learning Accurate Segmentation Purely from Self-Supervision [87.78965637247107]
  Selfment is a fully self-supervised framework that segments objects directly from raw images without human labels.<n>Selfment sets new state-of-the-art (SoTA) results across multiple benchmarks.
  arXiv  Detail & Related papers  (2026-02-27T07:36:32Z)
• LEFT: Learnable Fusion of Tri-view Tokens for Unsupervised Time Series Anomaly Detection [53.191369031661885]
  Unsupervised time series anomaly detection aims to build a model for identifying abnormal timestamps without assuming the availability of annotations.<n>We present Learnable Fusion of Tri-view Tokens (LEFT), a unified unsupervised TSAD framework that models anomalies as inconsistencies across complementary representations.<n>Experiments on real-world benchmarks show that LEFT yields the best detection accuracy against SOTA baselines, while achieving a 5x reduction on FLOPs and 8x speed-up for training.
  arXiv  Detail & Related papers  (2026-02-09T13:33:49Z)
• CrossCheck-Bench: Diagnosing Compositional Failures in Multimodal Conflict Resolution [20.823419395675412]
  CrossCheck-Bench is a diagnostic benchmark for evaluating contradiction detection in multimodal inputs.<n>We evaluate 13 state-of-the-art vision-language models and observe a consistent performance drop as tasks shift from perceptual matching to logical contradiction detection.
  arXiv  Detail & Related papers  (2025-11-19T12:17:15Z)
• Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts [49.99400612296149]
  We find that models can ace many benchmarks without strong visual understanding.<n>This is especially problematic for vision-centric benchmarks that are meant to require visual inputs.<n>We adopt a diagnostic principle for benchmark design: if a benchmark can be gamed, it will be gamed.
  arXiv  Detail & Related papers  (2025-11-06T18:43:21Z)
• PRISM-Bench: A Benchmark of Puzzle-Based Visual Tasks with CoT Error Detection [33.98107848014898]
  We introduce PRISM-Bench, a benchmark of puzzle-based visual challenges designed to evaluate multimodal large language models (MLLMs)<n>Unlike prior evaluations that measure only final-answer accuracy, PRISM-Bench introduces a diagnostic task: given a visual puzzle and a step-by-step chain-of-thought containing exactly one error, models must identify the first incorrect step.<n>The puzzles in PRISM-Bench require multi-step symbolic, geometric, and analogical reasoning, resisting shortcuts based on superficial pattern matching.
  arXiv  Detail & Related papers  (2025-10-27T17:57:52Z)
• OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding [50.72259772580637]
  We introduce OST-Bench, a benchmark designed to evaluate Online Spatio-Temporal understanding from the perspective of an agent.<n>Built on an efficient data collection pipeline, OST-Bench consists of 1.4k scenes and 10k question-answer pairs collected from ScanNet, Matterport3D, and ARKitScenes.<n>We find that both complex-based spatial reasoning demands and long-term memory retrieval requirements significantly drop model performance along two separate axes.
  arXiv  Detail & Related papers  (2025-07-10T17:56:07Z)
• Single-Stage Visual Relationship Learning using Conditional Queries [60.90880759475021]
  TraCQ is a new formulation for scene graph generation that avoids the multi-task learning problem and the entity pair distribution. We employ a DETR-based encoder-decoder conditional queries to significantly reduce the entity label space as well. Experimental results show that TraCQ not only outperforms existing single-stage scene graph generation methods, it also beats many state-of-the-art two-stage methods on the Visual Genome dataset.
  arXiv  Detail & Related papers  (2023-06-09T06:02:01Z)
• Learning to Track with Object Permanence [61.36492084090744]
  We introduce an end-to-end trainable approach for joint object detection and tracking. Our model, trained jointly on synthetic and real data, outperforms the state of the art on KITTI, and MOT17 datasets.
  arXiv  Detail & Related papers  (2021-03-26T04:43:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.