Related papers: 3D Reconstruction from Sketches

3D Reconstruction from Sketches

URL: http://arxiv.org/abs/2505.14621v1
Date: Tue, 20 May 2025 17:11:49 GMT
Title: 3D Reconstruction from Sketches
Authors: Abhimanyu Talwar, Julien Laasri,
Abstract summary: We consider the problem of reconstructing a 3D scene from multiple sketches.<n>We propose a pipeline which involves (1) stitching together multiple sketches through use of correspondence points, (2) converting the stitched sketch into a realistic image using a CycleGAN, and (3) estimating that image's depth-map using a pre-trained convolutional neural network based architecture called MegaDepth.
Score: 0.276240219662896
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: We consider the problem of reconstructing a 3D scene from multiple sketches. We propose a pipeline which involves (1) stitching together multiple sketches through use of correspondence points, (2) converting the stitched sketch into a realistic image using a CycleGAN, and (3) estimating that image's depth-map using a pre-trained convolutional neural network based architecture called MegaDepth. Our contribution includes constructing a dataset of image-sketch pairs, the images for which are from the Zurich Building Database, and sketches have been generated by us. We use this dataset to train a CycleGAN for our pipeline's second step. We end up with a stitching process that does not generalize well to real drawings, but the rest of the pipeline that creates a 3D reconstruction from a single sketch performs quite well on a wide variety of drawings.

Related papers

SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation [6.39528707908268]
There continues to be a lack of large-scale paired datasets for scene sketches. We propose a self-supervised method for scene sketch generation that does not rely on any existing scene sketch. We contribute a large-scale dataset centered around scene sketches, comprising highly semantically consistent "text-sketch-image" triplets.
arXiv Detail & Related papers (2024-05-29T06:43:49Z)
Sketch3D: Style-Consistent Guidance for Sketch-to-3D Generation [55.73399465968594]
This paper proposes a novel generation paradigm Sketch3D to generate realistic 3D assets with shape aligned with the input sketch and color matching the textual description. Three strategies are designed to optimize 3D Gaussians, i.e., structural optimization via a distribution transfer mechanism, color optimization with a straightforward MSE loss and sketch similarity optimization with a CLIP-based geometric similarity loss.
arXiv Detail & Related papers (2024-04-02T11:03:24Z)
Strokes2Surface: Recovering Curve Networks From 4D Architectural Design Sketches [0.0]
Strokes2Surface is an offline reconstruction pipeline that recovers well-connected curve networks from imprecise 4D sketches. Inspired by architectural sketching practices, our pipeline combines a classifier and two clustering models to achieve its goal.
arXiv Detail & Related papers (2023-06-12T16:26:38Z)
Make Your Brief Stroke Real and Stereoscopic: 3D-Aware Simplified Sketch to Portrait Generation [51.64832538714455]
Existing studies only generate portraits in the 2D plane with fixed views, making the results less vivid. In this paper, we present Stereoscopic Simplified Sketch-to-Portrait (SSSP), which explores the possibility of creating Stereoscopic 3D-aware portraits. Our key insight is to design sketch-aware constraints that can fully exploit the prior knowledge of a tri-plane-based 3D-aware generative model.
arXiv Detail & Related papers (2023-02-14T06:28:42Z)
DifferSketching: How Differently Do People Sketch 3D Objects? [78.44544977215918]
Multiple sketch datasets have been proposed to understand how people draw 3D objects. These datasets are often of small scale and cover a small set of objects or categories. We analyze the collected data at three levels, i.e., sketch-level, stroke-level, and pixel-level, under both spatial and temporal characteristics.
arXiv Detail & Related papers (2022-09-19T06:52:18Z)
SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling [75.957103837167]
Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch.
arXiv Detail & Related papers (2022-08-14T16:37:51Z)
Geometric Understanding of Sketches [0.0]
I explore two methods that help a system provide a geometric machine-understanding of sketches, and in-turn help a user accomplish a downstream task. The first work deals with interpretation of a 2D-line drawing as a graph structure, and also illustrates its effectiveness through its physical reconstruction by a robot. In the second work, we test the 3D-geometric understanding of a sketch-based system without explicit access to the information about 3D-geometry.
arXiv Detail & Related papers (2022-04-13T23:55:51Z)
Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches [65.96417928860039]
We use an encoder/decoder architecture for the sketch to mesh translation. We will show that this approach is easy to deploy, robust to style changes, and effective.
arXiv Detail & Related papers (2021-04-01T14:10:59Z)
SketchDesc: Learning Local Sketch Descriptors for Multi-view Correspondence [68.63311821718416]
We study the problem of multi-view sketch correspondence, where we take as input multiple freehand sketches with different views of the same object. This problem is challenging since the visual features of corresponding points at different views can be very different. We take a deep learning approach and learn a novel local sketch descriptor from data.
arXiv Detail & Related papers (2020-01-16T11:31:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.