TrackOcc: Camera-based 4D Panoptic Occupancy Tracking

• URL: http://arxiv.org/abs/2503.08471v1
• Date: Tue, 11 Mar 2025 14:17:06 GMT
• Title: TrackOcc: Camera-based 4D Panoptic Occupancy Tracking
• Abstract summary: Camera-based 4D Panoptic Occupancy Tracking simultaneously addresses panoptic occupancy segmentation and object tracking from camera-only input.<n>TrackOcc is a cutting-edge approach that processes image inputs in a streaming, end-to-end manner with 4D panoptic queries to address the proposed task.
• Score: 35.39897498797755
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Comprehensive and consistent dynamic scene understanding from camera input is essential for advanced autonomous systems. Traditional camera-based perception tasks like 3D object tracking and semantic occupancy prediction lack either spatial comprehensiveness or temporal consistency. In this work, we introduce a brand-new task, Camera-based 4D Panoptic Occupancy Tracking, which simultaneously addresses panoptic occupancy segmentation and object tracking from camera-only input. Furthermore, we propose TrackOcc, a cutting-edge approach that processes image inputs in a streaming, end-to-end manner with 4D panoptic queries to address the proposed task. Leveraging the localization-aware loss, TrackOcc enhances the accuracy of 4D panoptic occupancy tracking without bells and whistles. Experimental results demonstrate that our method achieves state-of-the-art performance on the Waymo dataset. The source code will be released at https://github.com/Tsinghua-MARS-Lab/TrackOcc.

Related papers

• 4DP-QA: Scalable QA for 4D Perception in Vision Language Models [68.67551474392373]
  We present a generation pipeline that focuses on motion-related scene understanding.<n>We take particular care of the entanglement of camera and object motion by casting tracking in both the traditional way and in a novel, fixed reference system, dubbed True-Motion Tracking.<n>From this pipeline, we generate a large-scale training dataset of 400K samples, 4DP-QA (4D Perception QA), and a 2.2K-sample benchmark, 4DP-QA-Bench.
  arXiv  Detail & Related papers  (2026-06-10T01:49:55Z)
• OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras [25.151688173881453]
  We present OccTrack360, a new benchmark for 4D panoptic occupancy tracking from surround-view fisheye cameras.<n>OccTrack360 provides substantially longer and more diverse sequences (1742234 frames) than prior benchmarks.<n>We also propose Focus on Sphere Occ, a framework that addresses two core challenges in fisheye occupancy tracking: distorted spherical projection and inaccurate voxel-space localization.
  arXiv  Detail & Related papers  (2026-03-09T15:55:26Z)
• Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking [17.16370461224889]
  4Dtemporal tracking is crucial for the safe and reliable operation of robots in dynamic environments.<n>In this paper we present Latent Gaussian splatting for 4D panoptic occupancy tracking.<n>We make code available at https://lags.cs.uni-freiburg.de/.
  arXiv  Detail & Related papers  (2026-02-26T16:34:49Z)
• Multi-View 3D Point Tracking [67.21282192436031]
  We introduce the first data-driven multi-view 3D point tracker, designed to track arbitrary points in dynamic scenes using multiple camera views.<n>Our model directly predicts 3D correspondences using a practical number of cameras.<n>We train on 5K synthetic multi-view Kubric sequences and evaluate on two real-world benchmarks.
  arXiv  Detail & Related papers  (2025-08-28T17:58:20Z)
• SpatialTrackerV2: 3D Point Tracking Made Easy [73.0350898700048]
  SpatialTrackerV2 is a feed-forward 3D point tracking method for monocular videos.<n>It decomposes world-space 3D motion into scene geometry, camera ego-motion, and pixel-wise object motion.<n>By learning geometry and motion jointly from such heterogeneous data, SpatialTrackerV2 outperforms existing 3D tracking methods by 30%.
  arXiv  Detail & Related papers  (2025-07-16T17:59:03Z)
• Easi3R: Estimating Disentangled Motion from DUSt3R Without Training [48.87063562819018]
  We introduce Easi3R, a simple yet efficient training-free method for 4D reconstruction. Our approach applies attention adaptation during inference, eliminating the need for from-scratch pre-training or network fine-tuning. Our experiments on real-world dynamic videos demonstrate that our lightweight attention adaptation significantly outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-31T17:59:58Z)
• DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild [85.03973683867797]
  This paper proposes a concise, elegant, and robust pipeline to estimate smooth camera trajectories and obtain dense point clouds for casual videos in the wild. We show that the proposed method achieves state-of-the-art performance in terms of camera pose estimation even in complex dynamic challenge scenes.
  arXiv  Detail & Related papers  (2024-11-20T13:01:16Z)
• DenseTrack: Drone-based Crowd Tracking via Density-aware Motion-appearance Synergy [33.57923199717605]
  Drone-based crowd tracking faces difficulties in accurately identifying and monitoring objects from an aerial perspective. To address these challenges, we present the Density-aware Tracking (DenseTrack) framework. DenseTrack capitalizes on crowd counting to precisely determine object locations, blending visual and motion cues to improve the tracking of small-scale objects.
  arXiv  Detail & Related papers  (2024-07-24T13:39:07Z)
• RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud [10.593320435411714]
  We introduce RaTrack, an innovative solution tailored for radar-based tracking. Our method focuses on motion segmentation and clustering, enriched by a motion estimation module. RaTrack showcases superior tracking precision of moving objects, largely surpassing the performance of the state of the art.
  arXiv  Detail & Related papers  (2023-09-18T13:02:29Z)
• Propagate And Calibrate: Real-time Passive Non-line-of-sight Tracking [84.38335117043907]
  We propose a purely passive method to track a person walking in an invisible room by only observing a relay wall. To excavate imperceptible changes in videos of the relay wall, we introduce difference frames as an essential carrier of temporal-local motion messages. To evaluate the proposed method, we build and publish the first dynamic passive NLOS tracking dataset, NLOS-Track.
  arXiv  Detail & Related papers  (2023-03-21T12:18:57Z)
• TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM [0.0]
  3D object tracking is a critical task in autonomous driving systems. In this paper we investigate the use of triplet embeddings in combination with motion representations for 3D object tracking.
  arXiv  Detail & Related papers  (2022-10-28T15:23:50Z)
• Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Tracking [53.64390261936975]
  We present Minkowski Tracker, a sparse-temporal R-CNN that jointly solves object detection and tracking problems. Inspired by region-based CNN (R-CNN), we propose to track motion as a second stage of the object detector R-CNN. We show in large-scale experiments that the overall performance gain of our method is due to four factors.
  arXiv  Detail & Related papers  (2022-08-22T04:47:40Z)
• EagerMOT: 3D Multi-Object Tracking via Sensor Fusion [68.8204255655161]
  Multi-object tracking (MOT) enables mobile robots to perform well-informed motion planning and navigation by localizing surrounding objects in 3D space and time. Existing methods rely on depth sensors (e.g., LiDAR) to detect and track targets in 3D space, but only up to a limited sensing range due to the sparsity of the signal. We propose EagerMOT, a simple tracking formulation that integrates all available object observations from both sensor modalities to obtain a well-informed interpretation of the scene dynamics.
  arXiv  Detail & Related papers  (2021-04-29T22:30:29Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)
• Integration of the 3D Environment for UAV Onboard Visual Object Tracking [7.652259812856325]
  Single visual object tracking from an unmanned aerial vehicle poses fundamental challenges. We introduce a pipeline that combines a model-free visual object tracker, a sparse 3D reconstruction, and a state estimator. By representing the position of the target in 3D space rather than in image space, we stabilize the tracking during ego-motion.
  arXiv  Detail & Related papers  (2020-08-06T18:37:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.