T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation

• URL: http://arxiv.org/abs/2606.30147v1
• Date: Mon, 29 Jun 2026 11:23:11 GMT
• Title: T2LDM++: A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation
• Abstract summary: Self-Conditioned Representation Guidance (SCRG) is used to alleviate object over-smoothing.<n>We construct two high-quality Text-LiDAR benchmarks using a generalized strategy of geometric annotations.<n>T2LDM++ can generate realistic LiDAR scenes with rich geometric details in unconditional and conditional settings.
• Score: 63.89237657631886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in Text-to-Image generation benefits from large-scale Text-Image pairs. However, the scarcity of Text-LiDAR pairs often causes over-smoothed scenes and limited controllability. In this paper, we rethink the limitations of Text-LiDAR generation task, focusing on alleviating insufficient training priors and constructing controllable Text-LiDAR data. We propose a \textbf{T}ext-\textbf{to}-\textbf{L}iDAR \textbf{D}iffusion \textbf{M}odel for LiDAR scene generation, T2LDM++, with a Self-Conditioned Representation Guidance (SCRG). Specifically, to alleviate object over-smoothing, SCRG employs a Guidance Network (GN) to provide reconstruction-based soft supervision to the Denoising Network (DN). This enables DN to learn geometry-aware representations through reconstruction guidance, leading to more accurate denoising in DDPMs. Meanwhile, through analysis and design, SCRG exhibits more effective and lightweight, while decoupled in inference, avoiding computational overhead. Furthermore, we construct two high-quality Text-LiDAR benchmarks ($>$100K samples) using a generalized strategy of geometric annotations, along with a controllability metric. Moreover, a directional position prior is designed to mitigate street distortion, further improving scene fidelity. Additionally, T2LDM++ supports multiple conditions, including (Semantic, Box, BEV, Camera)-to-LiDAR, Sparse-to-Dense, and Dense-to-Sparse generation, by learning a control encoder via frozen DN. With effective prior modeling and high-quality Text-LiDAR benchmarks, T2LDM++ can generate realistic LiDAR scenes with rich geometric details in unconditional and conditional settings.

Related papers

• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• Think Bright, Diffuse Nice: Enhancing T2I-ICL via Inductive-Bias Hint Instruction and Query Contrastive Decoding [10.961445998450008]
  Text-to-Image In-Context Learning enables customized image synthesis via interleaved text-image examples.<n>Existing methods rely on tailored training, which limits flexibility and raises deployment costs.<n>We propose TBDN, a training-free framework integrating two complementary closed-loop mechanisms.
  arXiv  Detail & Related papers  (2026-01-07T06:39:45Z)
• A Self-Conditioned Representation Guided Diffusion Model for Realistic Text-to-LiDAR Scene Generation [41.43267776407459]
  Text-to-LiDAR generation can customize 3D data with rich structures and diverse scenes for downstream tasks.<n>However, the scarcity of Text-LiDAR pairs often causes insufficient training priors, generating overly smooth 3D scenes.<n>We propose a Text-to-LiDAR Diffusion Model for scene generation, named T2LDM, with a Self-Conditioned Representation Guidance (SCRG)
  arXiv  Detail & Related papers  (2025-11-24T11:32:15Z)
• RAPO++: Cross-Stage Prompt Optimization for Text-to-Video Generation via Data Alignment and Test-Time Scaling [59.088798018184235]
  textbfRAPO++ is a cross-stage prompt optimization framework.<n>It unifies training-data-aligned refinement, test-time iterative scaling, and large language model fine-tuning.<n> RAPO++ achieves significant gains in semantic alignment, compositional reasoning, temporal stability, and physical plausibility.
  arXiv  Detail & Related papers  (2025-10-23T04:45:09Z)
• La La LiDAR: Large-Scale Layout Generation from LiDAR Data [45.5317990948996]
  Controllable generation of realistic LiDAR scenes is crucial for applications such as autonomous driving and robotics.<n>We propose Large-scale Layout-guided LiDAR generation model ("La La LiDAR"), a novel layout-guided generative framework.<n>La La LiDAR achieves state-of-the-art performance in both LiDAR generation and downstream perception tasks.
  arXiv  Detail & Related papers  (2025-08-05T17:59:55Z)
• LLMControl: Grounded Control of Text-to-Image Diffusion-based Synthesis with Multimodal LLMs [3.6016438645365834]
  We present a framework called LLM_Control to address the challenges of controllable T2I generation task.<n>By improving grounding capabilities, LLM_Control is introduced to accurately modulate the pre-trained diffusion models.<n>We utilize the multimodal LLM as a global controller to arrange spatial layouts, augment semantic descriptions and bind object attributes.
  arXiv  Detail & Related papers  (2025-07-26T12:57:02Z)
• LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting [53.58528891081709]
  We present LiDAR-GS, a real-time, high-fidelity re-simulation of LiDAR scans in public urban road scenes.<n>The method achieves state-of-the-art results in both rendering frame rate and quality on publically available large scene datasets.
  arXiv  Detail & Related papers  (2024-10-07T15:07:56Z)
• Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer [38.18396501696647]
  We propose Text2LiDAR, the first efficient, diverse, and text-controllable LiDAR data generation model. We design an equirectangular transformer architecture, utilizing the designed equirectangular attention to capture LiDAR features. We construct nuLiDARtext which offers diverse text descriptors for 34,149 LiDAR point clouds from 850 scenes.
  arXiv  Detail & Related papers  (2024-07-29T01:18:47Z)
• SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models [84.71887272654865]
  We present SparseCtrl to enable flexible structure control with temporally sparse signals. It incorporates an additional condition to process these sparse signals while leaving the pre-trained T2V model untouched. The proposed approach is compatible with various modalities, including sketches, depth maps, and RGB images.
  arXiv  Detail & Related papers  (2023-11-28T16:33:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.