SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery

• URL: http://arxiv.org/abs/2406.18151v2
• Date: Fri, 27 Sep 2024 03:36:47 GMT
• Title: SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery
• Authors: Jian Song, Hongruixuan Chen, Weihao Xuan, Junshi Xia, Naoto Yokoya,
• Abstract summary: Global semantic 3D understanding from single-view high-resolution remote sensing (RS) imagery is crucial for Earth Observation (EO) We develop a specialized synthetic data generation pipeline for EO and introduce SynRS3D, the largest synthetic RS 3D dataset. SynRS3D comprises 69,667 high-resolution optical images that cover six different city styles worldwide and feature eight land cover types, precise height information, and building change masks.
• Score: 17.364630812389038
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Global semantic 3D understanding from single-view high-resolution remote sensing (RS) imagery is crucial for Earth Observation (EO). However, this task faces significant challenges due to the high costs of annotations and data collection, as well as geographically restricted data availability. To address these challenges, synthetic data offer a promising solution by being easily accessible and thus enabling the provision of large and diverse datasets. We develop a specialized synthetic data generation pipeline for EO and introduce SynRS3D, the largest synthetic RS 3D dataset. SynRS3D comprises 69,667 high-resolution optical images that cover six different city styles worldwide and feature eight land cover types, precise height information, and building change masks. To further enhance its utility, we develop a novel multi-task unsupervised domain adaptation (UDA) method, RS3DAda, coupled with our synthetic dataset, which facilitates the RS-specific transition from synthetic to real scenarios for land cover mapping and height estimation tasks, ultimately enabling global monocular 3D semantic understanding based on synthetic data. Extensive experiments on various real-world datasets demonstrate the adaptability and effectiveness of our synthetic dataset and proposed RS3DAda method. SynRS3D and related codes will be available.

Related papers

• SyntheOcc: Synthesize Geometric-Controlled Street View Images through 3D Semantic MPIs [34.41011015930057]
  SyntheOcc addresses the challenge of how to efficiently encode 3D geometric information as conditional input to a 2D diffusion model. Our approach innovatively incorporates 3D semantic multi-plane images (MPIs) to provide comprehensive and spatially aligned 3D scene descriptions.
  arXiv  Detail & Related papers  (2024-10-01T02:29:24Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotations [55.022519020409405]
  This paper builds the first largest ever multi-modal 3D scene dataset and benchmark with hierarchical grounded language annotations, MMScan. The resulting multi-modal 3D dataset encompasses 1.4M meta-annotated captions on 109k objects and 7.7k regions as well as over 3.04M diverse samples for 3D visual grounding and question-answering benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T17:59:30Z)
• Hardness-Aware Scene Synthesis for Semi-Supervised 3D Object Detection [59.33188668341604]
  3D object detection serves as the fundamental task of autonomous driving perception. It is costly to obtain high-quality annotations for point cloud data. We propose a hardness-aware scene synthesis (HASS) method to generate adaptive synthetic scenes.
  arXiv  Detail & Related papers  (2024-05-27T17:59:23Z)
• Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [66.3814684757376]
  This work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level. The outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.
  arXiv  Detail & Related papers  (2024-03-21T10:38:18Z)
• SyntheWorld: A Large-Scale Synthetic Dataset for Land Cover Mapping and Building Change Detection [20.985372561774415]
  We present SyntheWorld, a synthetic dataset unparalleled in quality, diversity, and scale. It includes 40,000 images with submeter-level pixels and fine-grained land cover annotations of eight categories. We will release SyntheWorld to facilitate remote sensing image processing research.
  arXiv  Detail & Related papers  (2023-09-05T02:42:41Z)
• UniG3D: A Unified 3D Object Generation Dataset [75.49544172927749]
  UniG3D is a unified 3D object generation dataset constructed by employing a universal data transformation pipeline on ShapeNet datasets. This pipeline converts each raw 3D model into comprehensive multi-modal data representation. The selection of data sources for our dataset is based on their scale and quality.
  arXiv  Detail & Related papers  (2023-06-19T07:03:45Z)
• SCoDA: Domain Adaptive Shape Completion for Real Scans [78.92028595499245]
  3D shape completion from point clouds is a challenging task, especially from scans of real-world objects. We propose a new task, SCoDA, for the domain adaptation of real scan shape completion from synthetic data. We propose a novel cross-domain feature fusion method for knowledge transfer and a novel volume-consistent self-training framework for robust learning from real data.
  arXiv  Detail & Related papers  (2023-04-20T09:38:26Z)
• STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset [6.812704277866377]
  We introduce a synthetic aerial photogrammetry point clouds generation pipeline. Unlike generating synthetic data in virtual games, the proposed pipeline simulates the reconstruction process of the real environment. We present a richly-annotated synthetic 3D aerial photogrammetry point cloud dataset.
  arXiv  Detail & Related papers  (2022-03-17T03:50:40Z)
• Semi-synthesis: A fast way to produce effective datasets for stereo matching [16.602343511350252]
  Close-to-real-scene texture rendering is a key factor to boost up stereo matching performance. We propose semi-synthetic, an effective and fast way to synthesize large amount of data with close-to-real-scene texture. With further fine-tuning on the real dataset, we also achieve SOTA performance on Middlebury and competitive results on KITTI and ETH3D datasets.
  arXiv  Detail & Related papers  (2021-01-26T14:34:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.