SketchingReality: From Freehand Scene Sketches To Photorealistic Images

• URL: http://arxiv.org/abs/2602.14648v1
• Date: Mon, 16 Feb 2026 11:13:34 GMT
• Title: SketchingReality: From Freehand Scene Sketches To Photorealistic Images
• Authors: Ahmed Bourouis, Mikhail Bessmeltsev, Yulia Gryaditskaya,
• Abstract summary: We focus on balancing photorealism with sketch adherence when generating images from freehand input.<n>Key obstacle is the absence of ground-truth, pixel-aligned images.<n>We propose a modulation-based approach that prioritizes semantic interpretation of the sketch.<n>We show that our method outperforms existing approaches in both semantic alignment with freehand sketch inputs and in the realism and overall quality of the generated images.
• Score: 13.38976025870454
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent years have witnessed remarkable progress in generative AI, with natural language emerging as the most common conditioning input. As underlying models grow more powerful, researchers are exploring increasingly diverse conditioning signals, such as depth maps, edge maps, camera parameters, and reference images, to give users finer control over generation. Among different modalities, sketches are a natural and long-standing form of human communication, enabling rapid expression of visual concepts. Previous literature has largely focused on edge maps, often misnamed 'sketches', yet algorithms that effectively handle true freehand sketches, with their inherent abstraction and distortions, remain underexplored. We pursue the challenging goal of balancing photorealism with sketch adherence when generating images from freehand input. A key obstacle is the absence of ground-truth, pixel-aligned images: by their nature, freehand sketches do not have a single correct alignment. To address this, we propose a modulation-based approach that prioritizes semantic interpretation of the sketch over strict adherence to individual edge positions. We further introduce a novel loss that enables training on freehand sketches without requiring ground-truth pixel-aligned images. We show that our method outperforms existing approaches in both semantic alignment with freehand sketch inputs and in the realism and overall quality of the generated images.

Related papers

• SketchDreamer: Interactive Text-Augmented Creative Sketch Ideation [111.2195741547517]
  We present a method to generate controlled sketches using a text-conditioned diffusion model trained on pixel representations of images. Our objective is to empower non-professional users to create sketches and, through a series of optimisation processes, transform a narrative into a storyboard.
  arXiv  Detail & Related papers  (2023-08-27T19:44:44Z)
• Adapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval [85.39613457282107]
  Cross-domain nature of sketch-based image retrieval is challenging. We present an effective Adapt and Align'' approach to address the key challenges. Inspired by recent advances in image-text foundation models (e.g., CLIP) on zero-shot scenarios, we explicitly align the learned image embedding with a more semantic text embedding to achieve the desired knowledge transfer from seen to unseen classes.
  arXiv  Detail & Related papers  (2023-05-09T03:10:15Z)
• Picture that Sketch: Photorealistic Image Generation from Abstract Sketches [109.69076457732632]
  Given an abstract, deformed, ordinary sketch from untrained amateurs like you and me, this paper turns it into a photorealistic image. We do not dictate an edgemap-like sketch to start with, but aim to work with abstract free-hand human sketches. In doing so, we essentially democratise the sketch-to-photo pipeline, "picturing" a sketch regardless of how good you sketch.
  arXiv  Detail & Related papers  (2023-03-20T14:49:03Z)
• Text-Guided Scene Sketch-to-Photo Synthesis [5.431298869139175]
  We propose a method for scene-level sketch-to-photo synthesis with text guidance. To train our model, we use self-supervised learning from a set of photographs. Experiments show that the proposed method translates original sketch images that are not extracted from color images into photos with compelling visual quality.
  arXiv  Detail & Related papers  (2023-02-14T08:13:36Z)
• Semantics-Preserving Sketch Embedding for Face Generation [26.15479367792076]
  We introduce a novel W-W+ encoder architecture to take advantage of the high expressive power of W+ space. We also introduce an explicit intermediate representation for sketch semantic embedding. A novel sketch semantic interpretation approach is designed to automatically extract semantics from vectorized sketches.
  arXiv  Detail & Related papers  (2022-11-23T15:14:49Z)
• DeepPortraitDrawing: Generating Human Body Images from Freehand Sketches [75.4318318890065]
  We present DeepDrawing, a framework for converting roughly drawn sketches to realistic human body images. To encode complicated body shapes under various poses, we take a local-to-global approach. Our method produces more realistic images than the state-of-the-art sketch-to-image synthesis techniques.
  arXiv  Detail & Related papers  (2022-05-04T14:02:45Z)
• DeepFacePencil: Creating Face Images from Freehand Sketches [77.00929179469559]
  Existing image-to-image translation methods require a large-scale dataset of paired sketches and images for supervision. We propose DeepFacePencil, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches.
  arXiv  Detail & Related papers  (2020-08-31T03:35:21Z)
• Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn Sketches [133.01690754567252]
  Sketch-based image editing aims to synthesize and modify photos based on the structural information provided by the human-drawn sketches. Deep Plastic Surgery is a novel, robust and controllable image editing framework that allows users to interactively edit images using hand-drawn sketch inputs.
  arXiv  Detail & Related papers  (2020-01-09T08:57:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.