Physically-guided Image Generation for Multi-Projection Mapping

• URL: http://arxiv.org/abs/2606.22477v1
• Date: Sun, 21 Jun 2026 12:37:20 GMT
• Title: Physically-guided Image Generation for Multi-Projection Mapping
• Abstract summary: Projection Mapping (PM) enables seamless superimposition of digital content onto real-world 3D objects.<n>This paper formalizes two application-level generative paradigms: the cooperative paradigm and the adversarial paradigm.<n>We propose ConPhyG, a unified controllable generative multi-projection mapping framework.
• Score: 13.227477542441918
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Projection Mapping (PM) enables seamless superimposition of digital content onto real-world 3D objects, serving as a fundamental technique for immersive visualization, digital twins, and interactive art. Although text-to-image diffusion models have greatly facilitated customized content creation, directly integrating them into practical PM pipelines remains challenging due to the mismatch between idealized 2D generation and physical constraints. To bridge this gap, this paper formalizes two application-level generative paradigms: the cooperative paradigm (harmonizing generated semantics with physical attributes) and the adversarial paradigm (eliminating surface interference via radiometric compensation). Based on this, we propose ConPhyG, a unified controllable physically-guided generative multi-projection mapping framework that enables creators to interactively adjust physical constraints and flexibly switch generative paradigms. In cooperative mode, multi-dimensional physical priors (per-pixel gamut, depth, and edges) are injected into the diffusion process. In adversarial mode, the framework releases the generative potential and applies bounded numerical optimization for multi-projector radiometric compensation. It allows users to dynamically switch constraints to balance artistic freedom with physical feasibility. Furthermore, we extend ConPhyG to 360-degree multi-view consistent PM using a sequential generation strategy. Quantitative and qualitative evaluations on a real-world four-projector setup demonstrate that ConPhyG significantly outperforms state-of-the-art methods in geometric alignment, gamut utilization, and semantic fidelity.

Related papers

• Pixel-Space Diffusion Transformers [29.08838156660683]
  Latent diffusion models (LDMs) enable efficient high-resolution image synthesis by denoising in a VAE-compressed latent space.<n>This paper reviews Pixel-Space Diffusion Transformers (pDiTs) from the perspectives of model architecture, continuous generative mechanisms, and unified multimodal modeling.
  arXiv  Detail & Related papers  (2026-07-20T06:04:08Z)
• TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation [1.709500663078044]
  Trio is a pose-driven framework built upon the SD3.5M architecture.<n>It achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman.<n> Notably, it attains an AP of $64.33$ on Human-Art, representing a $30%$ improvement over prior arts.
  arXiv  Detail & Related papers  (2026-06-05T08:54:28Z)
• SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awareness [62.38746300197673]
  We propose a novel framework that internalizes 3D geometric awareness into unified image generation models.<n>Specifically, we first employ a Mixture-of-Transformers (MoT) architecture to augment the MLLM with a parallel spatial transformer.<n>By sharing self-attention with the MLLM, the spatial transformer learns to derive metric-depth maps of target images from rich semantic contexts.<n>These explicit geometric scaffolds are then injected into the diffusion backbone through a specialized depth adapter.
  arXiv  Detail & Related papers  (2026-04-29T06:46:59Z)
• MMPhysVideo: Scaling Physical Plausibility in Video Generation via Joint Multimodal Modeling [55.72785604682579]
  MMPhysVideo is a framework to scale physical plausibility in video generation in joint multimodal modeling.<n>MMPhysVideo consistently improves physical plausibility and visual quality over advanced models across various benchmarks.
  arXiv  Detail & Related papers  (2026-04-03T07:32:24Z)
• MMFace-DiT: A Dual-Stream Diffusion Transformer for High-Fidelity Multimodal Face Generation [2.5925656171325127]
  We introduce MMFace-DiT, a unified dual-stream diffusion transformer for synergistic multimodal face synthesis.<n>Its core novelty lies in a dual-stream transformer block that processes spatial (mask/sketch) and semantic (text) tokens in parallel.<n> MMFace-DiT achieves a 40% improvement in visual fidelity and prompt alignment over six state-of-the-art multimodal face generation models.
  arXiv  Detail & Related papers  (2026-03-30T21:44:18Z)
• StdGEN++: A Comprehensive System for Semantic-Decomposed 3D Character Generation [57.06461272772509]
  StdGEN++ is a novel and comprehensive system for generating high-fidelity, semantically decomposed 3D characters from diverse inputs.<n>It achieves state-of-the-art performance, significantly outperforming existing methods in geometric accuracy and semantic disentanglement.<n>The resulting structural independence unlocks advanced downstream capabilities, including non-destructive editing, physics-compliant animation, and gaze tracking.
  arXiv  Detail & Related papers  (2026-01-12T15:41:27Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidance [61.41904916189093]
  We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images.<n>We use hand-object interaction as geometric guidance to ensure plausible hand-object interactions.
  arXiv  Detail & Related papers  (2025-08-25T17:11:53Z)
• Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation [55.2480439325792]
  Safety-critical applications, such as autonomous driving and medical image analysis, require extensive multimodal data for rigorous testing.<n>This work introduces two novel methods for synthetic data generation in autonomous driving and medical image analysis, namely MObI and AnydoorMed, respectively.
  arXiv  Detail & Related papers  (2025-07-30T19:43:47Z)
• HF-VTON: High-Fidelity Virtual Try-On via Consistent Geometric and Semantic Alignment [22.960492450413497]
  We propose HF-VTON, a novel framework that ensures high-fidelity virtual try-on performance across diverse poses.<n> HF-VTON consists of three key modules: the Appearance-Preserving Warp Alignment Module, the Semantic Representation Module, and the Multimodal Prior-Guided Appearance Generation Generation Module.<n> Experimental results demonstrate that HF-VTON outperforms state-of-the-art methods on both VITON-HD and SAMP-VTONS.
  arXiv  Detail & Related papers  (2025-05-26T07:55:49Z)
• Guiding Diffusion with Deep Geometric Moments: Balancing Fidelity and Variation [35.428991756584935]
  We introduce Deep Geometric Moments (DGM) as a novel form of guidance that encapsulates the subject's visual features and nuances through a learned geometric prior.<n>Our experiments demonstrate that DGM effectively balance control and diversity in diffusion-based image generation, allowing a flexible control mechanism for steering the diffusion process.
  arXiv  Detail & Related papers  (2025-05-18T16:19:27Z)
• Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generation [54.588082888166504]
  We present Mogao, a unified framework that enables interleaved multi-modal generation through a causal approach.<n>Mogoo integrates a set of key technical improvements in architecture design, including a deep-fusion design, dual vision encoders, interleaved rotary position embeddings, and multi-modal classifier-free guidance.<n>Experiments show that Mogao achieves state-of-the-art performance in multi-modal understanding and text-to-image generation, but also excels in producing high-quality, coherent interleaved outputs.
  arXiv  Detail & Related papers  (2025-05-08T17:58:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.