MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation

• URL: http://arxiv.org/abs/2606.26087v1
• Date: Wed, 24 Jun 2026 17:56:33 GMT
• Title: MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation
• Abstract summary: We introduce MVTrack4Gen, a motion-aware training framework for camera-conditioning-only novel-view video diffusion models.<n>Our key finding is that specific attention layers encode strong correspondence cues, where query features attend to key features at geometrically corresponding locations.<n>By explicitly strengthening these motion-aware correspondences, MVTrack4Gen improves existing models to better follow the motion in the reference view and maintain cross-view geometric consistency.
• Score: 50.65653874052261
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthesizing a novel-view video from a monocular reference video along a target camera trajectory requires both geometric consistency and motion fidelity with respect to the reference video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in monocular videos. In contrast, camera-conditioning-only methods can achieve high visual quality but often struggle to preserve geometric and motion consistency. In this work, we introduce MVTrack4Gen (Multi-View point Tracking for Novel-View Generation), a motion-aware training framework that leverages multi-view point tracking as an additional geometric and motion supervision signal for camera-conditioning-only novel-view video diffusion models. Our key finding is that specific attention layers encode strong correspondence cues, where query features attend to key features at geometrically corresponding locations across views and over time, and the misalignment of these correspondences causes motion inconsistency. Based on this observation, we route these features into an auxiliary multi-view tracking head and jointly train the diffusion model with a point-tracking objective. By explicitly strengthening these motion-aware correspondences, MVTrack4Gen improves existing models to better follow the motion in the reference view and maintain cross-view geometric consistency. Across diverse benchmarks, our method achieves state-of-the-art geometric consistency and competitive camera accuracy.

Related papers

• Go-with-the-Track: Video Compositing and Motion Control with Point Tracking [50.22693773328114]
  Filmmaking demands precise motion control and reference image compositing.<n>We present Go-with-the-Track, which unifies both capabilities by jointly conditioning on multiple reference images and reference-anchored point-tracks.<n>Go-with-the-Track achieves superior motion and reference control in a single model.
  arXiv  Detail & Related papers  (2026-06-18T19:40:03Z)
• CalibAnyView: Beyond Single-View Camera Calibration in the Wild [59.66873936532375]
  Camera calibration is a fundamental prerequisite for reliable geometric perception.<n>Recent learning-based methods have shown promising results for single-view calibration, but inherently neglect geometric consistency across multiple views.<n>We introduce CalibAnyView, a unified formulation that supports an arbitrary number of input views.
  arXiv  Detail & Related papers  (2026-05-14T09:32:12Z)
• UniGeo: Unifying Geometric Guidance for Camera-Controllable Image Editing via Video Models [20.550231658763085]
  Camera-controllable image editing aims to synthesize novel views of a given scene under varying camera poses.<n>Existing methods rely on fragmented geometric guidance, such as only injecting point clouds at the representation level.<n>We propose UniGeo, a novel camera-controllable editing framework.
  arXiv  Detail & Related papers  (2026-04-19T18:11:08Z)
• Geometry-Aware Rotary Position Embedding for Consistent Video World Model [48.914346802616414]
  ViewRope is a geometry-aware encoding that injects camera-ray directions directly into video transformer self-attention layers.<n>Geometry-Aware Frame-Sparse Attention exploits these geometric cues to selectively attend to relevant historical frames.<n>Our results demonstrate that ViewRope substantially improves long-term consistency while reducing computational costs.
  arXiv  Detail & Related papers  (2026-02-08T08:01:16Z)
• SyncMV4D: Synchronized Multi-view Joint Diffusion of Appearance and Motion for Hand-Object Interaction Synthesis [47.61773799705708]
  We introduce SyncMV4D, the first model that jointly generates synchronized multi-view HOI videos and 4D motions.<n>Our method demonstrates superior performance to state-of-the-art alternatives in visual realism, motion plausibility, and multi-view consistency.
  arXiv  Detail & Related papers  (2025-11-24T17:14:19Z)
• Decoupling Dynamic Monocular Videos for Dynamic View Synthesis [50.93409250217699]
  We tackle the challenge of dynamic view synthesis from dynamic monocular videos in an unsupervised fashion. Specifically, we decouple the motion of the dynamic objects into object motion and camera motion, respectively regularized by proposed unsupervised surface consistency and patch-based multi-view constraints.
  arXiv  Detail & Related papers  (2023-04-04T11:25:44Z)
• ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild [57.37891682117178]
  We present a robust dense indirect structure-from-motion method for videos that is based on dense correspondence from pairwise optical flow. A novel neural network architecture is proposed for processing irregular point trajectory data. Experiments on MPI Sintel dataset show that our system produces significantly more accurate camera trajectories.
  arXiv  Detail & Related papers  (2022-07-19T09:19:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.