ObjChangeVR: Object State Change Reasoning from Continuous Egocentric Views in VR Environments

• URL: http://arxiv.org/abs/2603.06648v1
• Date: Fri, 27 Feb 2026 19:29:02 GMT
• Title: ObjChangeVR: Object State Change Reasoning from Continuous Egocentric Views in VR Environments
• Abstract summary: We introduce benchmarking for the question-answering task of object state change in virtual reality (VR)<n>We also propose ChangeVR, a framework that combines viewpoint-aware and temporal-based retrieval, along with cross-view reasoning.
• Score: 4.46498673219845
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in multimodal large language models (MLLMs) offer a promising approach for natural language-based scene change queries in virtual reality (VR). Prior work on applying MLLMs for object state understanding has focused on egocentric videos that capture the camera wearer's interactions with objects. However, object state changes may occur in the background without direct user interaction, lacking explicit motion cues and making them difficult to detect. Moreover, no benchmark exists for evaluating this challenging scenario. To address these challenges, we introduce ObjChangeVR-Dataset, specifically for benchmarking the question-answering task of object state change. We also propose ObjChangeVR, a framework that combines viewpoint-aware and temporal-based retrieval to identify relevant frames, along with cross-view reasoning that reconciles inconsistent evidence from multiple viewpoints. Extensive experiments demonstrate that ObjChangeVR significantly outperforms baseline approaches across multiple MLLMs.

Related papers

• Show Me When and Where: Towards Referring Video Object Segmentation in the Wild [98.87931411432106]
  In this work, we introduce a new setting towards in-the-wild RVOS.<n>Our new benchmark challenges RVOS methods to show not only where but also when objects appear in videos.<n>Our YoURVOS dataset offers an imperative benchmark, which will push forward the advancement of RVOS methods for practical applications.
  arXiv  Detail & Related papers  (2026-03-15T09:30:33Z)
• Vision-Motion-Reference Alignment for Referring Multi-Object Tracking via Multi-Modal Large Language Models [29.330083952817997]
  We propose a novel Vision-Motion-Reference aligned RMOT framework, named VMRMOT.<n>It integrates a motion modality extracted from object dynamics to enhance the alignment between vision modality and language references.<n>To the best of our knowledge, VMRMOT is the first approach to employ MLLMs in the RMOT task for vision-reference alignment.
  arXiv  Detail & Related papers  (2025-11-21T08:53:31Z)
• PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments [36.84821207878773]
  Visual reasoning in multimodal large language models (MLLMs) has primarily been studied in static, fully observable settings.<n>We introduce the Active Visual Reasoning (AVR) task, extending visual reasoning to partially observable, interactive environments.<n>We present a benchmark featuring multi-round interactive environments designed to assess both reasoning and information-gathering efficiency.
  arXiv  Detail & Related papers  (2025-10-24T02:59:00Z)
• InterRVOS: Interaction-aware Referring Video Object Segmentation [44.55538737075162]
  We introduce Interaction-aware Referring Video Object (InterRVOS), a novel task that focuses on the modeling of interactions.<n>It requires the model to segment the actor and target objects separately, reflecting their asymmetric roles in an interaction.<n>We present InterRVOS-127K, a large-scale dataset with over 127K automatically annotated expressions, including interaction expressions annotated with distinct masks for actor and target objects.
  arXiv  Detail & Related papers  (2025-06-03T01:16:13Z)
• Teaching VLMs to Localize Specific Objects from In-context Examples [56.797110842152]
  We find that present-day Vision-Language Models (VLMs) lack a fundamental cognitive ability: learning to localize specific objects in a scene by taking into account the context.<n>This work is the first to explore and benchmark personalized few-shot localization for VLMs.
  arXiv  Detail & Related papers  (2024-11-20T13:34:22Z)
• VOVTrack: Exploring the Potentiality in Videos for Open-Vocabulary Object Tracking [61.56592503861093]
  This issue amalgamates the complexities of open-vocabulary object detection (OVD) and multi-object tracking (MOT) Existing approaches to OVMOT often merge OVD and MOT methodologies as separate modules, predominantly focusing on the problem through an image-centric lens. We propose VOVTrack, a novel method that integrates object states relevant to MOT and video-centric training to address this challenge from a video object tracking standpoint.
  arXiv  Detail & Related papers  (2024-10-11T05:01:49Z)
• OSCaR: Object State Captioning and State Change Representation [52.13461424520107]
  This paper introduces the Object State Captioning and State Change Representation (OSCaR) dataset and benchmark. OSCaR consists of 14,084 annotated video segments with nearly 1,000 unique objects from various egocentric video collections. It sets a new testbed for evaluating multimodal large language models (MLLMs)
  arXiv  Detail & Related papers  (2024-02-27T01:48:19Z)
• OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog [10.290057801577662]
  OLViT is a novel model for video dialog operating over a multi-modal attention-based dialog state tracker. It maintains a global dialog state based on the output of an Object State Tracker (OST) and a Language State Tracker (LST)
  arXiv  Detail & Related papers  (2024-02-20T17:00:59Z)
• Scalable Video Object Segmentation with Identification Mechanism [125.4229430216776]
  This paper explores the challenges of achieving scalable and effective multi-object modeling for semi-supervised Video Object (VOS) We present two innovative approaches, Associating Objects with Transformers (AOT) and Associating Objects with Scalable Transformers (AOST) Our approaches surpass the state-of-the-art competitors and display exceptional efficiency and scalability consistently across all six benchmarks.
  arXiv  Detail & Related papers  (2022-03-22T03:33:27Z)
• Multi-modal Transformers Excel at Class-agnostic Object Detection [105.10403103027306]
  We argue that existing methods lack a top-down supervision signal governed by human-understandable semantics. We develop an efficient and flexible MViT architecture using multi-scale feature processing and deformable self-attention. We show the significance of MViT proposals in a diverse range of applications.
  arXiv  Detail & Related papers  (2021-11-22T18:59:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.