EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy

• URL: http://arxiv.org/abs/2605.24456v2
• Date: Tue, 26 May 2026 08:53:56 GMT
• Title: EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy
• Abstract summary: EgoProx is a benchmark for egocentric 3D proximity reasoning.<n>We organize our tasks along a cognitive chain, covering intention, exploration, exploitation, and chain-of-actions reasoning.
• Score: 21.116389181097254
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans constantly reason about 3D proximity, the relations between their body and surrounding objects, to guide perception and action in daily life. Whether multimodal large language models (MLLMs) can perform such embodied 3D reasoning remains unclear. To this end, we introduce EgoProx, a benchmark for egocentric 3D proximity reasoning. We organize our tasks along a cognitive chain, covering intention, exploration, exploitation, and chain-of-actions reasoning. We also design an agent based data engine that produces diverse and consistent QA pairs at scale. We benchmark prevailing MLLMs on EgoProx and conduct additional analyses with dataset specific and task specific instruction tuning. We observe large cross-domain gains, indicating that current MLLMs contain some spatial knowledge; however, they still struggle to effectively leverage it for spatial reasoning VQA.

Related papers

• EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs [23.973683864983272]
  EgoT-Bench is a fine-grained egocentric benchmark for grounded and verifiable operation-centric reasoning.<n>EgoT-Bench comprises 3,172 QA pairs over 351 egocentric retrospect videos.
  arXiv  Detail & Related papers  (2026-05-19T09:02:20Z)
• Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision [22.502853661316028]
  EgoPoint-Bench is a comprehensive question-answering benchmark designed to evaluate and enhance multimodal pointing reasoning in egocentric views.<n>We show that models fine-tuned on our synthetic data achieve significant performance gains and robust sim-to-real generalization.<n>This work highlights the importance of spatially aware supervision and offers a scalable path toward precise egocentric AI assistants.
  arXiv  Detail & Related papers  (2026-04-23T09:15:42Z)
• EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs [36.84953281346447]
  EgoMind is a Chainof-Thought framework that enables geometry-free spatial reasoning through RolePlay Caption.<n>With only 5K samples and 20K RL samples, EgoMind achieves effectiveness in MLLMBenchs, SITEBench, and SPBench.
  arXiv  Detail & Related papers  (2026-04-01T15:28:13Z)
• EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT [56.24624833924252]
  EgoThinker is a framework that endows MLs with robust egocentric reasoning capabilities through-temporal chain-of-thought supervision and a two-stage learning curriculum.<n>EgoThinker outperforms existing methods across multiple egocentric benchmarks, while achieving substantial improvements in fine-grained-temporal localization tasks.
  arXiv  Detail & Related papers  (2025-10-27T17:38:17Z)
• EgoCross: Benchmarking Multimodal Large Language Models for Cross-Domain Egocentric Video Question Answering [59.94048858464922]
  EgoCross is a benchmark to evaluate the cross-domain generalization of MLLMs in EgocentricQA.<n>EgoCross covers four diverse and challenging domains, including surgery, industry, extreme sports, and animal perspective.<n>It comprises approximately 1,000 QA pairs across 798 video clips, spanning four key QA tasks: prediction, recognition, localization, and counting.
  arXiv  Detail & Related papers  (2025-08-14T15:11:20Z)
• SIRI-Bench: Challenging VLMs' Spatial Intelligence through Complex Reasoning Tasks [51.774165536666864]
  We introduce SIRI-Bench, a benchmark designed to evaluate Vision-Language Models' structural spatial intelligence.<n>Bench comprises 9,000 video-question-answer triplets, where each problem is embedded in a realistic 3D scene.<n> Experimental results reveal that state-of-the-art VLMs struggle significantly on SIRI-Bench, underscoring the challenge of structural spatial reasoning.
  arXiv  Detail & Related papers  (2025-06-17T13:40:00Z)
• The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? [42.3970767778131]
  3D Large Language Models (LLMs) leveraging spatial information in point clouds for 3D spatial reasoning attract great attention.<n>Despite some promising results, the role of point clouds in 3D spatial reasoning remains under-explored.<n>We comprehensively evaluate and analyze these models to answer the research question: textitDoes point cloud truly boost the spatial reasoning capacities of 3D LLMs?
  arXiv  Detail & Related papers  (2025-04-06T16:38:48Z)
• EgoToM: Benchmarking Theory of Mind Reasoning from Egocentric Videos [26.930652137352197]
  We introduce EgoToM, a new video question-answering benchmark that extends Theory-of-Mind evaluation to egocentric domains.<n>Using a causal ToM model, we generate multi-choice video QA instances for the Ego4D dataset to benchmark the ability to predict a camera wearer's goals, beliefs, and next actions.<n>We study the performance of both humans and state of the art multimodal large language models (MLLMs) on these three interconnected inference problems.
  arXiv  Detail & Related papers  (2025-03-28T05:10:59Z)
• MLLM-For3D: Adapting Multimodal Large Language Model for 3D Reasoning Segmentation [91.94869042117621]
  Reasoning segmentation aims to segment target objects in complex scenes based on human intent and spatial reasoning.<n>Recent multimodal large language models (MLLMs) have demonstrated impressive 2D image reasoning segmentation.<n>We introduce MLLM-For3D, a framework that transfers knowledge from 2D MLLMs to 3D scene understanding.
  arXiv  Detail & Related papers  (2025-03-23T16:40:20Z)
• Exo2Ego: Exocentric Knowledge Guided MLLM for Egocentric Video Understanding [69.96199605596138]
  Current MLLMs primarily focus on third-person (exocentric) vision, overlooking the unique aspects of first-person (egocentric) videos.<n>We propose learning the mapping between exocentric and egocentric domains to enhance egocentric video understanding.<n>We introduce Ego-ExoClip, a pre-training dataset comprising 1.1M synchronized ego-exo clip-text pairs.
  arXiv  Detail & Related papers  (2025-03-12T08:10:33Z)
• Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model [51.83436609094658]
  We introduce Coarse Correspondences, a simple lightweight method that enhances MLLMs' spatial-temporal reasoning with 2D images as input. Our method uses a lightweight tracking model to identify primary object correspondences between frames in a video or across different image viewpoints. We demonstrate that this simple training-free approach brings substantial gains to GPT4-V/O consistently on four benchmarks.
  arXiv  Detail & Related papers  (2024-08-01T17:57:12Z)
• OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning [68.45848423501927]
  We propose a holistic vision-language dataset that aligns agent models with 3D driving tasks through counterfactual reasoning.<n>Our approach enhances decision-making by evaluating potential scenarios and their outcomes, similar to human drivers considering alternative actions.
  arXiv  Detail & Related papers  (2024-05-02T17:59:24Z)
• Think-Program-reCtify: 3D Situated Reasoning with Large Language Models [68.52240087262825]
  This work addresses the 3D situated reasoning task which aims to answer questions given egocentric observations in a 3D environment. We propose a novel framework that leverages the planning, tool usage, and reflection capabilities of large language models (LLMs) through a ThinkProgram-reCtify loop. Experiments and analysis on the SQA3D benchmark demonstrate the effectiveness, interpretability and robustness of our method.
  arXiv  Detail & Related papers  (2024-04-23T03:22:06Z)
• SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities [59.39858959066982]
  understanding and reasoning about spatial relationships is a fundamental capability for Visual Question Answering (VQA) and robotics. We develop an automatic 3D spatial VQA data generation framework that scales up to 2 billion VQA examples on 10 million real-world images. By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA.
  arXiv  Detail & Related papers  (2024-01-22T18:01:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.