LACONIC: A 3D Layout Adapter for Controllable Image Creation

• URL: http://arxiv.org/abs/2507.03257v2
• Date: Sat, 02 Aug 2025 03:59:32 GMT
• Title: LACONIC: A 3D Layout Adapter for Controllable Image Creation
• Authors: Léopold Maillard, Tom Durand, Adrien Ramanana Rahary, Maks Ovsjanikov,
• Abstract summary: Existing generative approaches for guided image synthesis rely on 2D controls in the image or text space.<n>We propose a novel conditioning approach, training method and adapter network that can be plugged into pretrained text-to-image diffusion models.<n>Our method supports camera control, conditioning geometries on explicit 3D and, for the first time, accounts for the entire context of a scene.
• Score: 22.96293773013579
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Existing generative approaches for guided image synthesis of multi-object scenes typically rely on 2D controls in the image or text space. As a result, these methods struggle to maintain and respect consistent three-dimensional geometric structure, underlying the scene. In this paper, we propose a novel conditioning approach, training method and adapter network that can be plugged into pretrained text-to-image diffusion models. Our approach provides a way to endow such models with 3D-awareness, while leveraging their rich prior knowledge. Our method supports camera control, conditioning on explicit 3D geometries and, for the first time, accounts for the entire context of a scene, i.e., both on and off-screen items, to synthesize plausible and semantically rich images. Despite its multi-modal nature, our model is lightweight, requires a reasonable number of data for supervised learning and shows remarkable generalization power. We also introduce methods for intuitive and consistent image editing and restyling, e.g., by positioning, rotating or resizing individual objects in a scene. Our method integrates well within various image creation workflows and enables a richer set of applications compared to previous approaches.

Related papers

• A Recipe for Generating 3D Worlds From a Single Image [28.396381735501524]
  We introduce a recipe for generating immersive 3D worlds from a single image.<n>This approach requires minimal training and uses existing generative models.<n>Tested on both synthetic and real images, our method produces high-quality 3D environments suitable for VR display.
  arXiv  Detail & Related papers  (2025-03-20T18:06:12Z)
• DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses [57.17501809717155]
  We present DreamDance, a novel method for animating human images using only skeleton pose sequences as conditional inputs.<n>Our key insight is that human images naturally exhibit multiple levels of correlation.<n>We construct the TikTok-Dance5K dataset, comprising 5K high-quality dance videos with detailed frame annotations.
  arXiv  Detail & Related papers  (2024-11-30T08:42:13Z)
• Towards Multi-View Consistent Style Transfer with One-Step Diffusion via Vision Conditioning [12.43848969320173]
  Stylized images from different viewpoints generated by our method achieve superior visual quality, with better structural integrity and less distortion. Our method effectively preserves the structural information and multi-view consistency in stylized images without any 3D information.
  arXiv  Detail & Related papers  (2024-11-15T12:02:07Z)
• 3D-free meets 3D priors: Novel View Synthesis from a Single Image with Pretrained Diffusion Guidance [61.06034736050515]
  We introduce a method capable of generating camera-controlled viewpoints from a single input image.<n>Our method excels in handling complex and diverse scenes without extensive training or additional 3D and multiview data.
  arXiv  Detail & Related papers  (2024-08-12T13:53:40Z)
• 3D Congealing: 3D-Aware Image Alignment in the Wild [44.254247801001675]
  3D Congealing is a problem of 3D-aware alignment for 2D images capturing semantically similar objects. We introduce a general framework that tackles the task without assuming shape templates, poses, or any camera parameters. Our framework can be used for various tasks such as correspondence matching, pose estimation, and image editing.
  arXiv  Detail & Related papers  (2024-04-02T17:32:12Z)
• Diffusion Models are Geometry Critics: Single Image 3D Editing Using Pre-Trained Diffusion Priors [24.478875248825563]
  We propose a novel image editing technique that enables 3D manipulations on single images. Our method directly leverages powerful image diffusion models trained on a broad spectrum of text-image pairs. Our method can generate high-quality 3D-aware image edits with large viewpoint transformations and high appearance and shape consistency with the input image.
  arXiv  Detail & Related papers  (2024-03-18T06:18:59Z)
• ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models [65.22994156658918]
  We present a method that learns to generate multi-view images in a single denoising process from real-world data. We design an autoregressive generation that renders more 3D-consistent images at any viewpoint.
  arXiv  Detail & Related papers  (2024-03-04T07:57:05Z)
• Guide3D: Create 3D Avatars from Text and Image Guidance [55.71306021041785]
  Guide3D is a text-and-image-guided generative model for 3D avatar generation based on diffusion models. Our framework produces topologically and structurally correct geometry and high-resolution textures.
  arXiv  Detail & Related papers  (2023-08-18T17:55:47Z)
• 3D-aware Image Generation using 2D Diffusion Models [23.150456832947427]
  We formulate the 3D-aware image generation task as multiview 2D image set generation, and further to a sequential unconditional-conditional multiview image generation process. We utilize 2D diffusion models to boost the generative modeling power of the method. We train our method on a large-scale dataset, i.e., ImageNet, which is not addressed by previous methods.
  arXiv  Detail & Related papers  (2023-03-31T09:03:18Z)
• SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation [89.47132156950194]
  We present a novel framework built to simplify 3D asset generation for amateur users. Our method supports a variety of input modalities that can be easily provided by a human. Our model can combine all these tasks into one swiss-army-knife tool.
  arXiv  Detail & Related papers  (2022-12-08T18:59:05Z)
• Self-Supervised Image Representation Learning with Geometric Set Consistency [50.12720780102395]
  We propose a method for self-supervised image representation learning under the guidance of 3D geometric consistency. Specifically, we introduce 3D geometric consistency into a contrastive learning framework to enforce the feature consistency within image views.
  arXiv  Detail & Related papers  (2022-03-29T08:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.