Seeing Isn't Orienting: A Cognitively Grounded Benchmark Reveals Systematic Orientation Failures in MLLMs Supplementary

• URL: http://arxiv.org/abs/2603.11410v1
• Date: Thu, 12 Mar 2026 00:52:16 GMT
• Title: Seeing Isn't Orienting: A Cognitively Grounded Benchmark Reveals Systematic Orientation Failures in MLLMs Supplementary
• Abstract summary: Current vision-language benchmarks largely conflate orientation with position and general scene understanding.<n>We introduce Discriminative Orientation Reasoning Intelligence (DORI), a hierarchical benchmark that makes object orientation the primary target.<n>DORI provides 33,656 multiple-choice questions covering 67 object categories in real-world and synthetic settings.
• Score: 24.852775714606224
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Humans learn object orientation progressively, from recognizing which way an object faces, to mentally rotating it, to reasoning about orientations between objects. Current vision-language benchmarks largely conflate orientation with position and general scene understanding. We introduce Discriminative Orientation Reasoning Intelligence (DORI), a cognitively grounded hierarchical benchmark that makes object orientation the primary target. Inspired by stages of human orientation cognition, DORI decomposes orientation into four dimensions, each evaluated at coarse (categorical) and granular (metric) levels. Composed from 13,652 images across 14 sources, DORI provides 33,656 multiple-choice questions covering 67 object categories in real-world and synthetic settings. Its coarse-to-granular design isolates orientation from confounds such as object recognition difficulty, scene clutter, and linguistic ambiguity via bounding-box isolation, standardized spatial reference frames, and structured prompts. Evaluating 24 state-of-the-art vision-language models shows a clear pattern: models that perform well on general spatial benchmarks are near-random on object-centric orientation tasks. The best models reach only 54.2% on coarse and 45.0% on granular judgments, with largest failures on compound rotations and shifts in inter-object reference frames. Large coarse-to-granular gaps reveal reliance on categorical heuristics rather than geometric reasoning, a limitation hidden by existing benchmarks. These results identify orientation understanding as an unsolved challenge for multimodal systems, with implications for robotic manipulation, 3D scene reconstruction, and human-AI interaction.

Related papers

• Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision [22.502853661316028]
  EgoPoint-Bench is a comprehensive question-answering benchmark designed to evaluate and enhance multimodal pointing reasoning in egocentric views.<n>We show that models fine-tuned on our synthetic data achieve significant performance gains and robust sim-to-real generalization.<n>This work highlights the importance of spatially aware supervision and offers a scalable path toward precise egocentric AI assistants.
  arXiv  Detail & Related papers  (2026-04-23T09:15:42Z)
• SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments [75.60795462502949]
  SpatialEvo is a self-evolving framework for 3D spatial reasoning.<n>It formalizes 16 spatial reasoning task categories under explicit geometric validation rules.<n>It converts unannotated 3D scenes into zero-noise interactive oracles, replacing model consensus with objective physical feedback.
  arXiv  Detail & Related papers  (2026-04-15T17:59:12Z)
• Prompt-Guided Spatial Understanding with RGB-D Transformers for Fine-Grained Object Relation Reasoning [7.670666668651702]
  We introduce a dedicated spatial reasoning framework for the Physical AI Spatial Intelligence Warehouse dataset introduced in the Track 3 2025 AI City Challenge.<n>Our approach enhances spatial comprehension by embedding mask dimensions in the form of bounding box coordinates directly into the input prompts.<n>Our comprehensive pipeline achieves a final score of 73.0606, placing 4th overall on the public leaderboard.
  arXiv  Detail & Related papers  (2025-10-13T22:51:20Z)
• Right Side Up? Disentangling Orientation Understanding in MLLMs with Fine-grained Multi-axis Perception Tasks [17.357441373079382]
  We introduce DORI (Discriminative Orientation Reasoning Intelligence), a benchmark establishing object orientation perception as a primary evaluation target.<n>DORI assesses four dimensions of orientation comprehension: frontal alignment, rotational transformations, relative directional relationships, and canonical orientation understanding.<n>Our evaluation of 15 state-of-the-art vision-language models reveals critical limitations.<n>DORI offers implications for improving robotic control, 3D scene reconstruction, and human-AI interaction in physical environments.
  arXiv  Detail & Related papers  (2025-05-27T18:22:44Z)
• Are We Done with Object-Centric Learning? [65.67948794110212]
  Object-centric learning (OCL) seeks to learn representations that only encode an object, isolated from other objects or background cues in a scene.<n>With recent sample-efficient segmentation models, we can separate objects in the pixel space and encode them independently.<n>We address the OOD generalization challenge caused by spurious background cues through the lens of OCL.
  arXiv  Detail & Related papers  (2025-04-09T17:59:05Z)
• Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMs [65.93003087656754]
  VisFactor is a benchmark that digitizes 20 vision-centric subtests from a well-established cognitive psychology assessment.<n>We evaluate 20 frontier Multimodal Large Language Models (MLLMs) from GPT, Gemini, Claude, LLaMA, Qwen, and SEED families.<n>The best-performing model achieves a score of only 25.19 out of 100, with consistent failures on tasks such as mental rotation, spatial relation inference, and figure-ground discrimination.
  arXiv  Detail & Related papers  (2025-02-23T04:21:32Z)
• SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation [50.060274413294586]
  We introduce the concept of semantic orientation, which defines object orientations using natural language.<n>By integrating semantic orientation into VLM agents, our SoFar framework enables 6-DoF spatial reasoning and generates robotic actions.
  arXiv  Detail & Related papers  (2025-02-18T18:59:02Z)
• SURDS: Benchmarking Spatial Understanding and Reasoning in Driving Scenarios with Vision Language Models [15.50826328938879]
  We introduce SURDS, a benchmark designed to evaluate the spatial reasoning capabilities of vision language models (VLMs)<n>Built on the nuScenes dataset, SURDS comprises 41,080 vision-question-answer training instances and 9,250 evaluation samples.<n>We propose a reinforcement learning-based alignment scheme leveraging spatially grounded reward signals.
  arXiv  Detail & Related papers  (2024-11-20T08:14:01Z)
• ZoomNeXt: A Unified Collaborative Pyramid Network for Camouflaged Object Detection [70.11264880907652]
  Recent object (COD) attempts to segment objects visually blended into their surroundings, which is extremely complex and difficult in real-world scenarios. We propose an effective unified collaborative pyramid network that mimics human behavior when observing vague images and camouflaged zooming in and out. Our framework consistently outperforms existing state-of-the-art methods in image and video COD benchmarks.
  arXiv  Detail & Related papers  (2023-10-31T06:11:23Z)
• SOOD: Towards Semi-Supervised Oriented Object Detection [57.05141794402972]
  This paper proposes a novel Semi-supervised Oriented Object Detection model, termed SOOD, built upon the mainstream pseudo-labeling framework. Our experiments show that when trained with the two proposed losses, SOOD surpasses the state-of-the-art SSOD methods under various settings on the DOTA-v1.5 benchmark.
  arXiv  Detail & Related papers  (2023-04-10T11:10:42Z)
• Benchmarking Unsupervised Object Representations for Video Sequences [111.81492107649889]
  We compare the perceptual abilities of four object-centric approaches: ViMON, OP3, TBA and SCALOR. Our results suggest that the architectures with unconstrained latent representations learn more powerful representations in terms of object detection, segmentation and tracking. Our benchmark may provide fruitful guidance towards learning more robust object-centric video representations.
  arXiv  Detail & Related papers  (2020-06-12T09:37:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.