Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds

• URL: http://arxiv.org/abs/2606.26964v2
• Date: Fri, 26 Jun 2026 09:10:44 GMT
• Title: Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds
• Abstract summary: Look-Before-Move is a camera planning framework that separates observation specification from motion execution.<n>We construct a dynamic 3D Story World Benchmark based on StoryBlender, covering 50 stories, 457 scenes, and 1585 shots with animated characters.<n>Experiments show that our framework improves subject perception, intent consistency, and trajectory quality over representative baselines.
• Score: 22.4160057556312
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As embodied AI and world models increasingly operate in dynamic 3D environments, visual perception must move beyond passively interpreting given observations toward actively deciding what to observe. We study this problem through camera planning in dynamic 3D story worlds, where the camera must not only generate smooth motion, but also decide what visual evidence should be acquired before it moves. We formulate this capability as Narrative-Grounded World Visual Attention, where the camera acts as an embodied observer that determines what to observe, how to compose the observation, and how to shift attention over time under narrative intent and physical 3D constraints. To realize this capability, we propose Look-Before-Move, a camera planning framework that separates observation specification from motion execution. It first builds a Semantic Observation Contract to convert directorial intent into executable visual constraints, then performs Monte Carlo Viewpoint Search to find narrative-compliant and geometrically feasible viewpoints, and finally applies Semantic Trajectory Grounding to connect selected viewpoints into continuous, collision-aware, and temporally coherent camera motion. We further construct a dynamic 3D Story World Benchmark based on StoryBlender, covering 50 stories, 457 scenes, and 1585 shots with animated characters, semantic scene configurations, and executable 3D environments. Experiments show that our framework improves subject perception, intent consistency, and trajectory quality over representative baselines, demonstrating the importance of organizing visual attention before generating camera motion.

Related papers

• Dyn-3D: Unveiling and Resolving Ego-Motion Ambiguity in Vision-Language Models [11.352994779962208]
  We introduce Dyn-3D, a benchmark using counterfactual 3D rendering to rigorously decouple visual changes from true kinematic properties.<n>By embedding metric physical ground truth into policy optimization, TempoVista explicitly grounds visual representations in 3D space.
  arXiv  Detail & Related papers  (2026-09-01T10:54:27Z)
• WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory [106.28234880163723]
  WorldDirector is a highly controllable video world model framework.<n>Our framework explicitly decouples semantic motion orchestration from visual generation.<n>We successfully preserve the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.
  arXiv  Detail & Related papers  (2026-07-02T17:59:59Z)
• LIME: Learning Intent-aware Camera Motion from Egocentric Video [56.023644847494836]
  We propose a vision-language camera-motion generator that combines an auto-regressive observation-gain output with a continuous flow-matching pose head.<n>We show that LIME can learn to actively choose camera poses from passive human video, turning ordinary egocentric recordings into supervision for intent-aware active perception.
  arXiv  Detail & Related papers  (2026-07-02T16:48:43Z)
• SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation [50.175987640001644]
  We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text.<n>We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes.
  arXiv  Detail & Related papers  (2026-07-02T06:29:18Z)
• 3D Scene-Adaptive Trajectory-Controllable Human Image Animation with Camera Movement [51.22216867938034]
  We present a 3D motion animation framework that controls both human motion trajectories within a reconstructed 3D environment for video generation.<n>We design a viewpoint-adaptive latent fusion mechanism to inject point-cloud geometric priors through scene-visibility masking into the generative process.<n>Experiments on two standard human image animation benchmark datasets demonstrate remarkable improvements of our method over the state of the arts in related video generation metics.
  arXiv  Detail & Related papers  (2026-06-29T16:22:54Z)
• Towards Spatio-Temporal World Scene Graph Generation from Monocular Videos [17.190262848866396]
  We introduce ActionGenome4D, a dataset that upgrades Action Genome videos into 4D scenes via feed-forward 3D reconstruction.<n>We then formalize World Scene Graph Generation (WSGG), the task of constructing a world scene graph at each timestamp.<n>We propose three complementary methods, each exploring a different inductive bias for reasoning about unobserved objects.
  arXiv  Detail & Related papers  (2026-03-13T17:18:03Z)
• Bringing Objects to Life: training-free 4D generation from 3D objects through view consistent noise [31.533802484121182]
  We introduce a training-free method for animating 3D objects by conditioning on textual prompts to guide 4D generation.<n>We first convert a 3D mesh into a 4D Neural Radiance Field (NeRF) that preserves the object's visual attributes.<n>Then, we animate the object using an Image-to-Video diffusion model driven by text.
  arXiv  Detail & Related papers  (2024-12-29T10:12:01Z)
• Decoupling Dynamic Monocular Videos for Dynamic View Synthesis [50.93409250217699]
  We tackle the challenge of dynamic view synthesis from dynamic monocular videos in an unsupervised fashion. Specifically, we decouple the motion of the dynamic objects into object motion and camera motion, respectively regularized by proposed unsupervised surface consistency and patch-based multi-view constraints.
  arXiv  Detail & Related papers  (2023-04-04T11:25:44Z)
• Neural Groundplans: Persistent Neural Scene Representations from a Single Image [90.04272671464238]
  We present a method to map 2D image observations of a scene to a persistent 3D scene representation. We propose conditional neural groundplans as persistent and memory-efficient scene representations.
  arXiv  Detail & Related papers  (2022-07-22T17:41:24Z)
• Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation [76.58256020932312]
  Estimating the motion of the camera together with the 3D structure of the scene from a monocular vision system is a complex task. We present a self-supervised learning framework for 3D object motion field estimation from monocular videos.
  arXiv  Detail & Related papers  (2021-10-13T16:45:01Z)
• Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structure [42.3091008598491]
  We develop a model that first estimates the latent 3D structure of the scene, including the segmentation of any moving objects. It then predicts future frames by simulating the object and camera dynamics, and rendering the resulting views. Experiments on two challenging datasets of natural videos show that our model can estimate 3D structure and motion segmentation from a single frame.
  arXiv  Detail & Related papers  (2021-06-16T18:00:12Z)
• Tracking Emerges by Looking Around Static Scenes, with Neural 3D Mapping [23.456046776979903]
  We propose to leverage multiview data of textitstatic points in arbitrary scenes (static or dynamic) to learn a neural 3D mapping module. The neural 3D mapper consumes RGB-D data as input, and produces a 3D voxel grid of deep features as output. We show that our unsupervised 3D object trackers outperform prior unsupervised 2D and 2.5D trackers, and approach the accuracy of supervised trackers.
  arXiv  Detail & Related papers  (2020-08-04T02:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.