OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras

• URL: http://arxiv.org/abs/2603.08521v1
• Date: Mon, 09 Mar 2026 15:55:26 GMT
• Title: OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras
• Abstract summary: We present OccTrack360, a new benchmark for 4D panoptic occupancy tracking from surround-view fisheye cameras.<n>OccTrack360 provides substantially longer and more diverse sequences (1742234 frames) than prior benchmarks.<n>We also propose Focus on Sphere Occ, a framework that addresses two core challenges in fisheye occupancy tracking: distorted spherical projection and inaccurate voxel-space localization.
• Score: 25.151688173881453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding dynamic 3D environments in a spatially continuous and temporally consistent manner is fundamental for robotics and autonomous driving. While recent advances in occupancy prediction provide a unified representation of scene geometry and semantics, progress in 4D panoptic occupancy tracking remains limited by the lack of benchmarks that support surround-view fisheye sensing, long temporal sequences, and instance-level voxel tracking. To address this gap, we present OccTrack360, a new benchmark for 4D panoptic occupancy tracking from surround-view fisheye cameras. OccTrack360 provides substantially longer and more diverse sequences (174~2234 frames) than prior benchmarks, together with principled voxel visibility annotations, including an all-direction occlusion mask and an MEI-based fisheye field-of-view mask. To establish a strong fisheye-oriented baseline, we further propose Focus on Sphere Occ (FoSOcc), a framework that addresses two core challenges in fisheye occupancy tracking: distorted spherical projection and inaccurate voxel-space localization. FoSOcc includes a Center Focusing Module (CFM) to enhance instance-aware spatial localization through supervised focus guidance, and a Spherical Lift Module (SLM) that extends perspective lifting to fisheye imaging under the Unified Projection Model. Extensive experiments on Occ3D-Waymo and OccTrack360 show that our method improves occupancy tracking quality with notable gains on geometrically regular categories, and establishes a strong baseline for future research on surround-view fisheye 4D occupancy tracking. The benchmark and source code will be made publicly available at https://github.com/YouthZest-Lin/OccTrack360.

Related papers

• TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views [50.26507994908593]
  We introduce TAPVid-MV (Tracking Any Point in Video across Multiple Views), the first benchmark for this setting.<n>It contains a curated set of 284 sequences, 1,142 calibrated camera streams, and 109,769 point tracks across seven subsets spanning indoor and outdoor domains.<n>We obtain these trajectories using dataset-specific auxiliary modalities: sensor depth, LiDAR, SLAM and SfM points, human meshes, posed object meshes, and simulation. Every sequence and trajectory is visually verified by human annotators.
  arXiv  Detail & Related papers  (2026-09-01T21:58:51Z)
• Streaming Gaussian Encoding for 4D Panoptic Occupancy Tracking [15.311099668839665]
  Camera-based 4D-POT is a promising paradigm for holistic scene understanding from multi-view imagery.<n>Recent mask-based pipelines achieve strong performance by propagating instance queries across frames.<n>We propose a streaming Gaussian encoder that maintains a persistent scene representation for 4D-POT.
  arXiv  Detail & Related papers  (2026-06-29T18:00:42Z)
• CylindTrack: Depth-Aware Cylindrical Motion Modeling for Panoramic Multi-Object Tracking [20.709779731310103]
  Multi-Object Tracking (MOT) is a core capability for embodied perception.<n> panoramic cameras are attractive for embodied systems because their 360 field of view reduces blind spots and keeps surrounding targets observable for longer durations.<n>CylindTrack is a depth-aware cylindrical tracking-by-detection framework for panoramic MOT.
  arXiv  Detail & Related papers  (2026-06-29T10:32:08Z)
• S3KF: Spherical State-Space Kalman Filtering for Panoramic 3D Multi-Object Tracking [24.79542959759503]
  Panoramic multi-object tracking is important for industrial safety monitoring, wide-area robotic perception, and infrastructure-light deployment in large workspaces.<n>Existing image-plane trackers are tightly coupled to the camera projection and become unreliable in panoramic imagery.<n>We present a panoramic 3D multi-object tracking framework built on a motorized rotating LiDAR and a quad-fisheye camera rig.
  arXiv  Detail & Related papers  (2026-03-29T06:01:29Z)
• Track4World: Feedforward World-centric Dense 3D Tracking of All Pixels [67.36972154532761]
  Estimating the 3D trajectory of every pixel from a monocular video is crucial and promising for a comprehensive understanding of the 3D dynamics of videos.<n>Recent monocular 3D tracking works demonstrate impressive performance, but are limited to either tracking sparse points on the first frame or a slow optimization-based framework for dense tracking.<n>We propose a feedforward model, called Track4World, enabling an efficient holistic 3D tracking of every pixel in the world-centric coordinate system.
  arXiv  Detail & Related papers  (2026-03-03T03:45:43Z)
• Latent Gaussian Splatting for 4D Panoptic Occupancy Tracking [17.16370461224889]
  4Dtemporal tracking is crucial for the safe and reliable operation of robots in dynamic environments.<n>In this paper we present Latent Gaussian splatting for 4D panoptic occupancy tracking.<n>We make code available at https://lags.cs.uni-freiburg.de/.
  arXiv  Detail & Related papers  (2026-02-26T16:34:49Z)
• Multi-View 3D Point Tracking [67.21282192436031]
  We introduce the first data-driven multi-view 3D point tracker, designed to track arbitrary points in dynamic scenes using multiple camera views.<n>Our model directly predicts 3D correspondences using a practical number of cameras.<n>We train on 5K synthetic multi-view Kubric sequences and evaluate on two real-world benchmarks.
  arXiv  Detail & Related papers  (2025-08-28T17:58:20Z)
• GRASPTrack: Geometry-Reasoned Association via Segmentation and Projection for Multi-Object Tracking [11.436294975354556]
  GRASPTrack is a novel MOT framework that integrates monocular depth estimation and instance segmentation into a standard TBD pipeline.<n>These 3D point clouds are then voxelized to enable a precise and robust Voxel-Based 3D Intersection-over-Union.
  arXiv  Detail & Related papers  (2025-08-11T15:56:21Z)
• SpatialTrackerV2: 3D Point Tracking Made Easy [73.0350898700048]
  SpatialTrackerV2 is a feed-forward 3D point tracking method for monocular videos.<n>It decomposes world-space 3D motion into scene geometry, camera ego-motion, and pixel-wise object motion.<n>By learning geometry and motion jointly from such heterogeneous data, SpatialTrackerV2 outperforms existing 3D tracking methods by 30%.
  arXiv  Detail & Related papers  (2025-07-16T17:59:03Z)
• TrackOcc: Camera-based 4D Panoptic Occupancy Tracking [35.39897498797755]
  Camera-based 4D Panoptic Occupancy Tracking simultaneously addresses panoptic occupancy segmentation and object tracking from camera-only input.<n>TrackOcc is a cutting-edge approach that processes image inputs in a streaming, end-to-end manner with 4D panoptic queries to address the proposed task.
  arXiv  Detail & Related papers  (2025-03-11T14:17:06Z)
• DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstruction [65.46359561104867]
  We target the challenge of online 2D and 3D point tracking from unposed monocular camera input.<n>We leverage 3D Gaussian splatting to reconstruct dynamic scenes in an online fashion.<n>We aim to inspire the community to advance online point tracking and reconstruction, expanding the applicability to diverse real-world scenarios.
  arXiv  Detail & Related papers  (2024-09-03T17:58:03Z)
• BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View [54.48052449493636]
  3D Single Object Tracking (SOT) is a fundamental task in computer vision and plays a critical role in applications like autonomous driving.<n>We propose BEVTrack, a simple yet effective motion-based tracking method.<n>We show that BEVTrack achieves state-of-the-art results while operating at 200 FPS, enabling real-time applicability.
  arXiv  Detail & Related papers  (2023-09-05T12:42:26Z)
• Monocular Quasi-Dense 3D Object Tracking [99.51683944057191]
  A reliable and accurate 3D tracking framework is essential for predicting future locations of surrounding objects and planning the observer's actions in numerous applications such as autonomous driving. We propose a framework that can effectively associate moving objects over time and estimate their full 3D bounding box information from a sequence of 2D images captured on a moving platform.
  arXiv  Detail & Related papers  (2021-03-12T15:30:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.