Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models

• URL: http://arxiv.org/abs/2608.25418v1
• Date: Wed, 26 Aug 2026 06:20:06 GMT
• Title: Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models
• Abstract summary: LiDAR-SAM2 is a framework that turns a 2D video foundation, model SAM2, into a scalable source of supervision for the 4D LiDAR domain.<n>On the data side, it automatically generates temporally coherent LiDAR-level labels from SAM2 video masks.<n>On the modeling side, a tailored modality interface and a two-stage learning adapt objective SAM2s video segmentation kernel to-temporal LiDAR structure.
• Score: 51.635812333029314
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Progress in 4D LiDAR segmentation is bottlenecked by data. Assigning temporally consistent labels across sparse point cloud sequences is costly and hard to scale, and every new task or domain tends to demand fresh dense annotation. This motivates a simple question of whether high-quality LiDAR training data can be produced automatically, without any human labeling. To this end, we introduce LiDAR-SAM2, a framework that turns a 2D video foundation model, SAM2, into a scalable source of supervision for the 4D LiDAR domain. On the data side, it automatically generates temporally coherent LiDAR-level labels from SAM2 video masks through multi-view projection and spatio-temporal aggregation. On the modeling side, a tailored modality interface and a two-stage learning objective adapt SAM2's video segmentation kernel to spatio-temporal LiDAR structure, so that a single click per object yields a consistent mask track across the sequence. Trained with no human LiDAR annotation, LiDAR-SAM2 produces semantic and panoptic labels on SemanticKITTI that approach the quality of full human annotation from only a few points, and models trained on these labels approach the performance of full ground-truth supervision. This positions LiDAR-SAM2 as a scalable labeling tool that substantially reduces the annotation burden for 3D and 4D scene understanding.

Related papers

• ALISE: Annotation-Free LiDAR Instance Segmentation for Autonomous Driving [9.361724251990154]
  We introduce ALISE, a novel framework that performs LiDAR instance segmentation without any annotations.<n>Our approach starts by employing Vision Foundation Models (VFMs), guided by text and images, to produce initial pseudo-labels.<n>We then refine these labels through a dedicated manual-temporal voting module, which combines 2D and 3D semantics for both offline and online optimization.<n>This comprehensive design results in significant performance gains, establishing a new state-of-the-art for unsupervised 3D instance segmentation.
  arXiv  Detail & Related papers  (2025-10-07T10:15:18Z)
• Learning to Generate 4D LiDAR Sequences [28.411253849111755]
  We present LiDARCrafter, a unified framework that converts free-form language into editable LiDAR sequences.<n>LiDARCrafter achieves state-of-the-art fidelity, controllability, and temporal consistency, offering a foundation for LiDAR-based simulation and data augmentation.
  arXiv  Detail & Related papers  (2025-09-15T14:14:48Z)
• LiDARCrafter: Dynamic 4D World Modeling from LiDAR Sequences [28.411253849111755]
  LiDARCrafter is a unified framework for 4D LiDAR generation and editing.<n>It achieves state-of-the-art performance in fidelity, controllability, and temporal consistency across all levels.<n>The code and benchmark are released to the community.
  arXiv  Detail & Related papers  (2025-08-05T17:59:56Z)
• SALT: A Flexible Semi-Automatic Labeling Tool for General LiDAR Point Clouds with Cross-Scene Adaptability and 4D Consistency [12.47241966988188]
  We propose a flexible Semi-Automatic Labeling Tool (SALT) for general LiDAR point clouds.<n>SALT operates directly on raw LiDAR data, automatically generating pre-segmentation results.<n>We anticipate that SALT's open-sourcing will catalyze substantial expansion of current LiDAR datasets.
  arXiv  Detail & Related papers  (2025-03-31T11:46:55Z)
• SuperFlow++: Enhanced Spatiotemporal Consistency for Cross-Modal Data Pretraining [62.433137130087445]
  SuperFlow++ is a novel framework that integrates pretraining and downstream tasks using consecutive camera pairs.<n>We show that SuperFlow++ outperforms state-of-the-art methods across diverse tasks and driving conditions.<n>With strong generalizability and computational efficiency, SuperFlow++ establishes a new benchmark for data-efficient LiDAR-based perception in autonomous driving.
  arXiv  Detail & Related papers  (2025-03-25T17:59:57Z)
• OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving [74.06413946934002]
  We introduce OLiDM, a novel framework capable of generating high-fidelity LiDAR data at both the object and the scene levels.<n>OLiDM consists of two pivotal components: the Object-Scene Progressive Generation (OPG) module and the Object Semantic Alignment (OSA) module.<n>OPG adapts to user-specific prompts to generate desired foreground objects, which are subsequently employed as conditions in scene generation.<n>OSA aims to rectify the misalignment between foreground objects and background scenes, enhancing the overall quality of the generated objects.
  arXiv  Detail & Related papers  (2024-12-23T02:43:29Z)
• Segment, Lift and Fit: Automatic 3D Shape Labeling from 2D Prompts [50.181870446016376]
  This paper proposes an algorithm for automatically labeling 3D objects from 2D point or box prompts. Unlike previous arts, our auto-labeler predicts 3D shapes instead of bounding boxes and does not require training on a specific dataset.
  arXiv  Detail & Related papers  (2024-07-16T04:53:28Z)
• Approaching Outside: Scaling Unsupervised 3D Object Detection from 2D Scene [22.297964850282177]
  We propose LiDAR-2D Self-paced Learning (LiSe) for unsupervised 3D detection. RGB images serve as a valuable complement to LiDAR data, offering precise 2D localization cues. Our framework devises a self-paced learning pipeline that incorporates adaptive sampling and weak model aggregation strategies.
  arXiv  Detail & Related papers  (2024-07-11T14:58:49Z)
• An Effective Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds [50.19288542498838]
  3D single object tracking in LiDAR point clouds (LiDAR SOT) plays a crucial role in autonomous driving. Current approaches all follow the Siamese paradigm based on appearance matching. We introduce a motion-centric paradigm to handle LiDAR SOT from a new perspective.
  arXiv  Detail & Related papers  (2023-03-21T17:28:44Z)
• LiDAR-based 4D Panoptic Segmentation via Dynamic Shifting Network [56.71765153629892]
  We propose the Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. Our proposed DS-Net achieves superior accuracies over current state-of-the-art methods in both tasks. We extend DS-Net to 4D panoptic LiDAR segmentation by the temporally unified instance clustering on aligned LiDAR frames.
  arXiv  Detail & Related papers  (2022-03-14T15:25:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.