SM4RT: Learning Structured Motion Geometry for 4D Reconstruction

• URL: http://arxiv.org/abs/2607.22534v1
• Date: Fri, 24 Jul 2026 17:59:51 GMT
• Title: SM4RT: Learning Structured Motion Geometry for 4D Reconstruction
• Abstract summary: Most existing motion perception methods treat motion as independent point-wise displacements, ignoring the structured nature of physical motion.<n>We propose SM4RT, a Structured Motion 4D Reconstruction Transformer for end-to-end 3D reconstruction and structured motion perception.<n> SM4RT achieves strong motion reconstruction performance while preserving the geometric structure of scene motion.
• Score: 67.18230176143263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geometry Foundation Models (GFMs) have substantially advanced monocular 3D reconstruction, yet extending this capability to 4D dynamic understanding remains a fundamental challenge. Most existing motion perception methods (e.g., sparse tracking, dense point-wise flow) treat motion as independent point-wise displacements, ignoring the structured nature of physical motion. However, real-world objects usually obey rigid-body kinematics, and points thus usually move collectively, not in isolation. Motion itself possesses geometric structure: physical objects undergo a set of rigid-body transformations governed by SE(3), rather than unstructured point-wise displacements. Building on this insight, we propose SM4RT, a Structured Motion 4D Reconstruction Transformer for end-to-end 3D reconstruction and structured motion perception. SM4RT introduces Structure-of-Motion to represent scene dynamics, where scene motion is decomposed into a compact set of motion bases, each represented as a temporal sequence of 6D twists in SE(3). Dense scene motion is then recovered by sparse, time-shared per-pixel assignment weights over these bases, ensuring points on the same object share a common rigid-body motion trajectory. SM4RT introduces a parallel motion geometry encoder and decoder that jointly infer 3D geometry, world-coordinate motion, and scene kinematic structure in a single forward pass from monocular RGB video. SM4RT achieves strong motion reconstruction performance while preserving the geometric structure of scene motion.

Related papers

• MotionScale: Reconstructing Appearance, Geometry, and Motion of Dynamic Scenes with Scalable 4D Gaussian Splatting [54.859943475818234]
  MotionScale is a 4D Gaussian Splatting framework that scales efficiently to large scenes and extended sequences.<n>Experiments on challenging real-world benchmarks demonstrate that MotionScale significantly outperforms state-of-the-art methods in both reconstruction quality and temporal stability.
  arXiv  Detail & Related papers  (2026-03-31T06:03:59Z)
• Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraints [21.83046776294786]
  Articulat3D is a novel framework that constructs digital twins from casually captured monocular videos.<n>We first propose Motion Prior-Driven Initialization, which leverages 3D point tracks to exploit the low-dimensional structure of articulated motion.<n>We then introduce Geometric and Motion Constraints Refinement, which enforces physically plausible articulation.<n>Experiments demonstrate that Articulat3D achieves state-of-the-art performance on synthetic benchmarks and real-world casually captured monocular videos.
  arXiv  Detail & Related papers  (2026-03-12T06:59:44Z)
• Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow [61.297800738187355]
  Flow4R predicts a minimal per-pixel property set-3D point position, scene flow, pose weight, and confidence-from two-view inputs using a Vision Transformer.<n> trained jointly on static and dynamic datasets, Flow4R achieves state-of-the-art performance on 4D reconstruction and tracking tasks.
  arXiv  Detail & Related papers  (2026-02-15T06:58:08Z)
• Motion 3-to-4: 3D Motion Reconstruction for 4D Synthesis [53.48281548500864]
  Motion 3-to-4 is a feed-forward framework for synthesising high-quality 4D dynamic objects from a single monocular video.<n>Our model learns a compact motion latent representation and predicts per-frame trajectories to recover complete robustness, temporally coherent geometry.
  arXiv  Detail & Related papers  (2026-01-20T18:59:48Z)
• 4D Primitive-Mâché: Glueing Primitives for Persistent 4D Scene Reconstruction [28.50411933478524]
  We present a dynamic reconstruction system that receives a casual monocular RGB video as input, and outputs a persistent reconstruction of the scene.<n>In other words, we reconstruct not only the the currently visible parts of the scene, but also all previously viewed parts, which enables replaying the complete reconstruction across all timesteps.
  arXiv  Detail & Related papers  (2025-12-18T14:06:15Z)
• C4D: 4D Made from 3D through Dual Correspondences [77.04731692213663]
  We introduce C4D, a framework that leverages temporal correspondences to extend existing 3D reconstruction formulation to 4D.<n>C4D captures two types of correspondences: short-term optical flow and long-term point tracking.<n>We train a dynamic-aware point tracker that provides additional mobility information.
  arXiv  Detail & Related papers  (2025-10-16T17:59:06Z)
• Shape of Motion: 4D Reconstruction from a Single Video [42.42669078777769]
  We introduce a method for reconstructing generic dynamic scenes, featuring explicit, persistent 3D motion trajectories in the world coordinate frame.<n>First, we exploit the low-dimensional structure of 3D motion by representing scene motion with a compact set of SE(3) motion bases.<n>Second, we take advantage of off-the-shelf data-driven priors such as monocular depth maps and long-range 2D tracks, and devise a method to effectively consolidate these noisy supervisory signals.
  arXiv  Detail & Related papers  (2024-07-18T17:59:08Z)
• Class-agnostic Reconstruction of Dynamic Objects from Videos [127.41336060616214]
  We introduce REDO, a class-agnostic framework to REconstruct the Dynamic Objects from RGBD or calibrated videos. We develop two novel modules. First, we introduce a canonical 4D implicit function which is pixel-aligned with aggregated temporal visual cues. Second, we develop a 4D transformation module which captures object dynamics to support temporal propagation and aggregation.
  arXiv  Detail & Related papers  (2021-12-03T18:57:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.