Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation

• URL: http://arxiv.org/abs/2604.18468v1
• Date: Mon, 20 Apr 2026 16:20:57 GMT
• Title: Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation
• Abstract summary: Asset Harvester is an image-to-3D model and end-to-end pipeline that converts sparse, in-thewild object observations from real driving logs into complete, simulation-ready assets.<n>SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges.
• Score: 63.01022057888141
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Closed-loop simulation is a core component of autonomous vehicle (AV) development, enabling scalable testing, training, and safety validation before real-world deployment. Neural scene reconstruction converts driving logs into interactive 3D environments for simulation, but it does not produce complete 3D object assets required for agent manipulation and large-viewpoint novel-view synthesis. To address this challenge, we present Asset Harvester, an image-to-3D model and end-to-end pipeline that converts sparse, in-the-wild object observations from real driving logs into complete, simulation-ready assets. Rather than relying on a single model component, we developed a system-level design for real-world AV data that combines large-scale curation of object-centric training tuples, geometry-aware preprocessing across heterogeneous sensors, and a robust training recipe that couples sparse-view-conditioned multiview generation with 3D Gaussian lifting. Within this system, SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges. Together with hybrid data curation, augmentation, and self-distillation, this system enables scalable conversion of sparse AV object observations into reusable 3D assets.

Related papers

• Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Control [86.64250947574749]
  Reconstructing articulated 3D objects is important for animation, gaming, and robotic simulations.<n>Recent neural networks can estimate the articulated structure of 3D objects, but their generalization remains limited by the scarcity of annotated data.<n>We introduce Instruct-Particulate, a model that takes a 3D mesh together with a target kinematic specification.
  arXiv  Detail & Related papers  (2026-06-12T17:59:36Z)
• Generative Texture Diversification of 3D Pedestrians for Robust Autonomous Driving Perception [0.0]
  In this paper, we propose a method for scaling variability in 3D pedestrian assets for synthetic scene generation.<n>We generate multiple distinct pedestrian instances by synthesizing diverse facial textures and identity-level appearance variations.<n>Using the assets, we construct synthetic datasets and study the impact of mixing real and synthetic data for RGB-based object detection.
  arXiv  Detail & Related papers  (2026-05-13T16:35:50Z)
• GenAssets: Generating in-the-wild 3D Assets in Latent Space [31.627626425629035]
  High-quality 3D assets for traffic participants are critical for multi-sensor simulation.<n>Building assets from in-the-wild data is key for diversity and realism.<n>We propose a 3D latent diffusion model that learns on in-the-wild LiDAR and camera data.
  arXiv  Detail & Related papers  (2026-04-24T20:56:55Z)
• SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLM [14.374484080984423]
  We propose SIMART, a unified MLLM framework that jointly performs part-level decomposition and kinematic prediction.<n>By introducing a Sparse 3D VQ-VAE, SIMART reduces token counts by 70% vs. dense voxel tokens, enabling high-fidelity multi-part assemblies.
  arXiv  Detail & Related papers  (2026-03-24T16:16:52Z)
• URDF-Anything+: Autoregressive Articulated 3D Models Generation for Physical Simulation [45.4820195450296]
  We present an end-to-end autoregressive framework that directly generates executable articulated object models from visual observations.<n>Building on this capability, we enable a new Real-Follow-Sim paradigm, where high-fidelity digital twins constructed from visual observations allow policies trained in simulation to transfer to real robots without online adaptation.
  arXiv  Detail & Related papers  (2026-03-14T16:23:44Z)
• SCPainter: A Unified Framework for Realistic 3D Asset Insertion and Novel View Synthesis [3.614325475261039]
  3D asset insertion and novel view synthesis (NVS) are key components for autonomous driving simulation, enhancing the diversity of training data.<n>We present SCPainter, a unified framework which integrates 3D Splat (GS) car asset representations and 3D scene point clouds with diffusion-based generation.<n>The 3D GS assets and 3D scene point clouds are projected together into novel views, and these projections are used to condition a diffusion model to generate high quality images.
  arXiv  Detail & Related papers  (2025-12-27T21:28:48Z)
• Particulate: Feed-Forward 3D Object Articulation [89.78788418174946]
  Particulate is a feed-forward approach that, given a single static 3D mesh of an everyday object, directly infers all attributes of the underlying articulated structure.<n>We train the network end-to-end on a diverse collection of articulated 3D assets from public datasets.<n>During inference, Particulate lifts the network's feed-forward prediction to the input mesh, yielding a fully articulated 3D model in seconds.
  arXiv  Detail & Related papers  (2025-12-12T18:59:51Z)
• R3D2: Realistic 3D Asset Insertion via Diffusion for Autonomous Driving Simulation [78.26308457952636]
  This paper introduces R3D2, a lightweight, one-step diffusion model designed to overcome limitations in autonomous driving simulation.<n>It enables realistic insertion of complete 3D assets into existing scenes by generating plausible rendering effects-such as shadows and consistent lighting-in real time.<n>We show that R3D2 significantly enhances the realism of inserted assets, enabling use-cases like text-to-3D asset insertion and cross-scene/dataset object transfer.
  arXiv  Detail & Related papers  (2025-06-09T14:50:19Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance [76.7746870349809]
  We present ComboVerse, a 3D generation framework that produces high-quality 3D assets with complex compositions by learning to combine multiple models. Our proposed framework emphasizes spatial alignment of objects, compared with standard score distillation sampling.
  arXiv  Detail & Related papers  (2024-03-19T03:39:43Z)
• Transferable Active Grasping and Real Embodied Dataset [48.887567134129306]
  We show how to search for feasible viewpoints for grasping by the use of hand-mounted RGB-D cameras. A practical 3-stage transferable active grasping pipeline is developed, that is adaptive to unseen clutter scenes. In our pipeline, we propose a novel mask-guided reward to overcome the sparse reward issue in grasping and ensure category-irrelevant behavior.
  arXiv  Detail & Related papers  (2020-04-28T08:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.