WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Image

• URL: http://arxiv.org/abs/2506.23518v2
• Date: Wed, 06 Aug 2025 08:53:51 GMT
• Title: WAVE: Warp-Based View Guidance for Consistent Novel View Synthesis Using a Single Image
• Authors: Jiwoo Park, Tae Eun Choi, Youngjun Jun, Seong Jae Hwang,
• Abstract summary: This paper proposes a novel view-consistent image generation method which utilizes diffusion models without additional modules.<n>Our key idea is to enhance diffusion models with a training-free method that enables adaptive attention manipulation and noise reinitialization.<n>Our method improves view consistency across various diffusion models, demonstrating its broader applicability.
• Score: 3.4248731707266264
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generating high-quality novel views of a scene from a single image requires maintaining structural coherence across different views, referred to as view consistency. While diffusion models have driven advancements in novel view synthesis, they still struggle to preserve spatial continuity across views. Diffusion models have been combined with 3D models to address the issue, but such approaches lack efficiency due to their complex multi-step pipelines. This paper proposes a novel view-consistent image generation method which utilizes diffusion models without additional modules. Our key idea is to enhance diffusion models with a training-free method that enables adaptive attention manipulation and noise reinitialization by leveraging view-guided warping to ensure view consistency. Through our comprehensive metric framework suitable for novel-view datasets, we show that our method improves view consistency across various diffusion models, demonstrating its broader applicability.

Related papers

• Synthesizing Consistent Novel Views via 3D Epipolar Attention without Re-Training [102.82553402539139]
  Large diffusion models demonstrate remarkable zero-shot capabilities in novel view synthesis from a single image.<n>These models often face challenges in maintaining consistency across novel and reference views.<n>We propose to use epipolar geometry to locate and retrieve overlapping information from the input view.<n>This information is then incorporated into the generation of target views, eliminating the need for training or fine-tuning.
  arXiv  Detail & Related papers  (2025-02-25T14:04:22Z)
• Merging and Splitting Diffusion Paths for Semantically Coherent Panoramas [33.334956022229846]
  We propose the Merge-Attend-Diffuse operator, which can be plugged into different types of pretrained diffusion models used in a joint diffusion setting. Specifically, we merge the diffusion paths, reprogramming self- and cross-attention to operate on the aggregated latent space. Our method maintains compatibility with the input prompt and visual quality of the generated images while increasing their semantic coherence.
  arXiv  Detail & Related papers  (2024-08-28T09:22:32Z)
• MultiDiff: Consistent Novel View Synthesis from a Single Image [60.04215655745264]
  MultiDiff is a novel approach for consistent novel view synthesis of scenes from a single RGB image. Our results demonstrate that MultiDiff outperforms state-of-the-art methods on the challenging, real-world datasets RealEstate10K and ScanNet.
  arXiv  Detail & Related papers  (2024-06-26T17:53:51Z)
• Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models [23.786473791344395]
  Cross-attention layers in diffusion models tend to disproportionately focus on certain tokens during the generation process. We introduce attention regulation, an on-the-fly optimization approach at inference time to align attention maps with the input text prompt. Experiment results show that our method consistently outperforms other baselines.
  arXiv  Detail & Related papers  (2024-03-11T02:18:27Z)
• ViewFusion: Towards Multi-View Consistency via Interpolated Denoising [48.02829400913904]
  We introduce ViewFusion, a training-free algorithm that can be seamlessly integrated into existing pre-trained diffusion models. Our approach adopts an auto-regressive method that implicitly leverages previously generated views as context for the next view generation. Our framework successfully extends single-view conditioned models to work in multiple-view conditional settings without any additional fine-tuning.
  arXiv  Detail & Related papers  (2024-02-29T04:21:38Z)
• Multi-View Unsupervised Image Generation with Cross Attention Guidance [23.07929124170851]
  This paper introduces a novel pipeline for unsupervised training of a pose-conditioned diffusion model on single-category datasets. We identify object poses by clustering the dataset through comparing visibility and locations of specific object parts. Our model, MIRAGE, surpasses prior work in novel view synthesis on real images.
  arXiv  Detail & Related papers  (2023-12-07T14:55:13Z)
• Layered Rendering Diffusion Model for Controllable Zero-Shot Image Synthesis [15.76266032768078]
  This paper introduces innovative solutions to enhance spatial controllability in diffusion models reliant on text queries.<n>We first introduce vision guidance as a foundational spatial cue within the perturbed distribution.<n>We propose a universal framework, Layered Rendering Diffusion (LRDiff), which constructs an image-rendering process with multiple layers.
  arXiv  Detail & Related papers  (2023-11-30T10:36:19Z)
• Consistent123: Improve Consistency for One Image to 3D Object Synthesis [74.1094516222327]
  Large image diffusion models enable novel view synthesis with high quality and excellent zero-shot capability. These models have no guarantee of view consistency, limiting the performance for downstream tasks like 3D reconstruction and image-to-3D generation. We propose Consistent123 to synthesize novel views simultaneously by incorporating additional cross-view attention layers and the shared self-attention mechanism.
  arXiv  Detail & Related papers  (2023-10-12T07:38:28Z)
• Deceptive-NeRF/3DGS: Diffusion-Generated Pseudo-Observations for High-Quality Sparse-View Reconstruction [60.52716381465063]
  We introduce Deceptive-NeRF/3DGS to enhance sparse-view reconstruction with only a limited set of input images. Specifically, we propose a deceptive diffusion model turning noisy images rendered from few-view reconstructions into high-quality pseudo-observations. Our system progressively incorporates diffusion-generated pseudo-observations into the training image sets, ultimately densifying the sparse input observations by 5 to 10 times.
  arXiv  Detail & Related papers  (2023-05-24T14:00:32Z)
• Consistent View Synthesis with Pose-Guided Diffusion Models [51.37925069307313]
  Novel view synthesis from a single image has been a cornerstone problem for many Virtual Reality applications. We propose a pose-guided diffusion model to generate a consistent long-term video of novel views from a single image.
  arXiv  Detail & Related papers  (2023-03-30T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.