DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs

• URL: http://arxiv.org/abs/2607.12503v2
• Date: Wed, 15 Jul 2026 03:49:06 GMT
• Title: DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs
• Abstract summary: 4D-temporal reasoning is essential for understanding dynamic worlds and enabling embodied interaction.<n>Current models rely on sparse frame-level observations, fragmenting continuous dynamic cues.<n>We propose DynTrace, a training-free framework for 4D-temporal reasoning with two complementary components.
• Score: 41.4243802276804
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 4D spatio-temporal reasoning, jointly modeling 3D spatial structure and temporal evolution, is essential for understanding dynamic worlds and enabling embodied interaction. While current Multimodal Large Language Models (MLLMs) show strong capabilities in static scene understanding and coarse-grained 4D tasks, they still have notable limitations in continuous dynamic scene perception, especially in tracking dynamic object evidence for coherent 4D spatio-temporal reasoning. This shortcoming stems mainly from relying on sparse frame-level observations, fragmenting continuous dynamic cues and leaving models unable to disentangle genuine object dynamics from camera-induced apparent motion. Inspired by humans tracking dynamic cues while compensating for viewpoint changes, we propose DynTrace, a training-free framework for 4D spatio-temporal reasoning with two complementary components. Dynamic Trajectory Visualization (DTV) reprojects world-coordinate trajectories onto the image plane, providing geometry-informed visual priors that disentangle genuine object dynamics from camera-induced apparent motion. Meanwhile, the Dynamic Trace Token (DT-Token), organized into a Dynamic Trace Graph (DTG), tracks object-level dynamic cues, trace evolution, and key moments, maintaining continuous dynamic object evidence for coherent 4D reasoning. Together, these two components equip MLLMs with continuously tracked dynamic object evidence, grounded in geometry-informed visual priors and structured spatio-temporal traces. DynTrace consistently improves open-source MLLMs, achieving state-of-the-art results on Dyn-Bench, VLM4D, and DSI-Bench, validating the importance of tracking dynamic object evidence for robust 4D spatio-temporal reasoning.

Related papers

• IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer [64.5765465030666]
  We propose IGGT4D, a streaming instance-grounded geometry Transformer for online 4D scene understanding.<n> IGGT4D processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally updates a unified representation of camera motion, geometry, and object identity.<n> Experiments on 3D reconstruction, pose estimation, instance spatial tracking, and open-vocabulary segmentation demonstrate that IGGT4D outperforms existing streaming baselines.
  arXiv  Detail & Related papers  (2026-07-21T16:00:01Z)
• WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory [106.28234880163723]
  WorldDirector is a highly controllable video world model framework.<n>Our framework explicitly decouples semantic motion orchestration from visual generation.<n>We successfully preserve the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.
  arXiv  Detail & Related papers  (2026-07-02T17:59:59Z)
• LMM-Track4D: Eliciting 4D Dynamic Reasoning in LMMs via Trajectory-Grounded Dialogue [4.481491966887467]
  Recent models (LMMs) have become increasingly capable on image video understanding, yet still struggle to sustain 4D continuous dynamic reasoning.<n>To study this capability, we propose a new task in which a model must answer queries while returning 3D trajectories over an entire short or a specified segment of a longer clip.<n>Experiments on Track4-Track show consistent improvements over strong baselines, suggesting that dynamic state modeling is a useful design principle for eliciting 4D dynamic reasoning in LMMs.
  arXiv  Detail & Related papers  (2026-05-19T05:35:13Z)
• Syn4D: A Multiview Synthetic 4D Dataset [92.50132053229817]
  3D reconstruction and tracking of dynamic scenes from monocular video remains an important open challenge in computer vision.<n>To address this limitation, we introduce Syn4D, a multiview synthetic dynamic scenes.<n>Key feature of Syn4D is the ability to unproject any pixel into 3D.
  arXiv  Detail & Related papers  (2026-05-06T17:59:55Z)
• Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D World [49.80040477190479]
  Humans inhabit a physical 4D world where geometric structure and semantic content evolve over time.<n>We introduce Dyn-Bench, a large-scale benchmark built from diverse real-world and synthetic video datasets.<n>We find that existing models cannot simultaneously maintain strong performance in both-temporal reasoning and dynamic object grounding.
  arXiv  Detail & Related papers  (2026-03-13T07:42:16Z)
• DynamicVGGT: Learning Dynamic Point Maps for 4D Scene Reconstruction in Autonomous Driving [36.56424549984779]
  We propose a unified feed-forward framework that extends VGGT from static 3D perception to dynamic 4D reconstruction.<n>Our goal is to model point motion within feed-forward 3D models in a dynamic and temporally coherent manner.<n>Experiments on autonomous driving datasets demonstrate that DynamicVGGT significantly outperforms existing methods in reconstruction accuracy.
  arXiv  Detail & Related papers  (2026-03-09T11:24:14Z)
• The Dynamic Prior: Understanding 3D Structures for Casual Dynamic Videos [19.25337083769716]
  We introduce the Dynamic Prior (ourmodel) to robustly identify dynamic objects without task-specific training.<n>ourmodel can be seamlessly integrated into state-of-the-art pipelines for camera pose optimization, depth reconstruction, and 4D trajectory estimation.
  arXiv  Detail & Related papers  (2025-12-05T03:31:49Z)
• C4D: 4D Made from 3D through Dual Correspondences [77.04731692213663]
  We introduce C4D, a framework that leverages temporal correspondences to extend existing 3D reconstruction formulation to 4D.<n>C4D captures two types of correspondences: short-term optical flow and long-term point tracking.<n>We train a dynamic-aware point tracker that provides additional mobility information.
  arXiv  Detail & Related papers  (2025-10-16T17:59:06Z)
• Understanding Dynamic Scenes in Ego Centric 4D Point Clouds [7.004204907286336]
  EgoDynamic4D is a novel QA benchmark on highly dynamic scenes.<n>We design 12 dynamic QA tasks covering agent motion, human-object interaction prediction, relation, understanding trajectory, temporal-causal reasoning, with fine-grained, explicit metrics.<n>Our method consistently outperforms baselines, validating the effectiveness of multimodal temporal modeling for ego-centric dynamic scene understanding.
  arXiv  Detail & Related papers  (2025-08-10T09:08:04Z)
• MotionTrack: Learning Motion Predictor for Multiple Object Tracking [68.68339102749358]
  We introduce a novel motion-based tracker, MotionTrack, centered around a learnable motion predictor. Our experimental results demonstrate that MotionTrack yields state-of-the-art performance on datasets such as Dancetrack and SportsMOT.
  arXiv  Detail & Related papers  (2023-06-05T04:24:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.