Point2Pose: Occlusion-Recovering 6D Pose Tracking and 3D Reconstruction for Multiple Unknown Objects Via 2D Point Trackers

• URL: http://arxiv.org/abs/2604.10415v1
• Date: Sun, 12 Apr 2026 02:17:49 GMT
• Title: Point2Pose: Occlusion-Recovering 6D Pose Tracking and 3D Reconstruction for Multiple Unknown Objects Via 2D Point Trackers
• Abstract summary: Point2Pose is a model-free method for causal 6D pose tracking of rigid objects from monocular RGB-D video.<n>Our approach tracks multiple unseen objects without requiring object CAD models or category priors.<n> Experiments show that Point2Pose achieves performance comparable to the state-of-the-art methods on a severe-occlusion benchmark.
• Score: 17.68066197994233
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present Point2Pose, a model-free method for causal 6D pose tracking of multiple rigid objects from monocular RGB-D video. Initialized only from sparse image points on the objects to be tracked, our approach tracks multiple unseen objects without requiring object CAD models or category priors. Point2Pose leverages a 2D point tracker to obtain long-range correspondences, enabling instant recovery after complete occlusion. Simultaneously, the system incrementally reconstructs an online Truncated Signed Distance Function (TSDF) representation of the tracked targets. Alongside the method, we introduce a new multi-object tracking dataset comprising both simulation and real-world sequences, with motion-capture ground truth for evaluation. Experiments show that Point2Pose achieves performance comparable to the state-of-the-art methods on a severe-occlusion benchmark, while additionally supporting multi-object tracking and recovery from complete occlusion, capabilities that are not supported by previous model-free tracking approaches.

Related papers

• TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views [50.26507994908593]
  We introduce TAPVid-MV (Tracking Any Point in Video across Multiple Views), the first benchmark for this setting.<n>It contains a curated set of 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks across seven subsets spanning indoor and outdoor domains.<n>We obtain these trajectories using dataset-specific auxiliary modalities: sensor depth, LiDAR, SLAM and SfM points, human meshes, posed object meshes, and simulation. Every sequence and trajectory is visually verified by human annotators.
  arXiv  Detail & Related papers  (2026-09-01T21:58:51Z)
• RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes [55.15267980622371]
  RRTrack is an efficient, recoverable object 6D pose tracker.<n>It enables robust tracking through fast motion and target disappearance--reappearance.<n>Experiments show RRTrack improves equal-subset mean ADD-S AR by 66.3% and ADD-S AUC by 65.7% over FoundationPose.
  arXiv  Detail & Related papers  (2026-07-26T14:05:46Z)
• SpatialTrackerV2: 3D Point Tracking Made Easy [73.0350898700048]
  SpatialTrackerV2 is a feed-forward 3D point tracking method for monocular videos.<n>It decomposes world-space 3D motion into scene geometry, camera ego-motion, and pixel-wise object motion.<n>By learning geometry and motion jointly from such heterogeneous data, SpatialTrackerV2 outperforms existing 3D tracking methods by 30%.
  arXiv  Detail & Related papers  (2025-07-16T17:59:03Z)
• Street Gaussians without 3D Object Tracker [84.89933388445185]
  Most existing methods rely on labor-intensive manual labeling of object poses.<n>We propose a stable object tracking module by leveraging associations from 2D deep trackers within a 3D object fusion strategy.<n>We address inevitable tracking errors by further introducing a motion learning strategy in an implicit feature space that autonomously corrects trajectory errors and recovers missed detections.
  arXiv  Detail & Related papers  (2024-12-07T05:49:42Z)
• Tracking by 3D Model Estimation of Unknown Objects in Videos [122.56499878291916]
  We argue that this representation is limited and instead propose to guide and improve 2D tracking with an explicit object representation. Our representation tackles a complex long-term dense correspondence problem between all 3D points on the object for all video frames. The proposed optimization minimizes a novel loss function to estimate the best 3D shape, texture, and 6DoF pose.
  arXiv  Detail & Related papers  (2023-04-13T11:32:36Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation [8.854112907350624]
  3D multi-object tracking plays a vital role in autonomous navigation. Many approaches detect objects in 2D RGB sequences for tracking, which is lack of reliability when localizing objects in 3D space. We propose a novel convolutional operation, named RelationConv, to better exploit the correlation between each pair of objects in the adjacent frames.
  arXiv  Detail & Related papers  (2020-11-25T16:14:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.