Text-to-Image Generation for Projector-Camera System Registration

• URL: http://arxiv.org/abs/2607.03046v1
• Date: Fri, 03 Jul 2026 07:37:21 GMT
• Title: Text-to-Image Generation for Projector-Camera System Registration
• Abstract summary: We propose a method based on deep neural networks to generate a single natural image from text-based prompts.<n>Our approach aims to generate a single natural image from text-based prompts that not only appears realistic but also possesses rich spatial features.<n>Our trained network predicts the correspondence between projector and camera images, significantly improving registration accuracy across various procam configurations.
• Score: 18.826226391746193
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Establishing correspondence between projector and camera images in a procam (projector + camera) system is essential for achieving high-resolution pixel matching, referred to as procam registration. The highest accuracy is typically obtained using structured light patterns (e.g., stripes or blobs). However, these methods are often inefficient and lack meaningful information for human viewers. Although some have explored the use of natural images, these often fail to provide a sufficient distribution of features to achieve comparable accuracy. Additionally, existing methods struggle to cope with environmental factors such as surface textures and variations in brightness due to ambient light or changes in camera exposure. To address these limitations, we propose a method based on deep neural networks. Our approach aims to generate a single natural image from text-based prompts that not only appears realistic but also possesses rich spatial features to enhance registration accuracy in procam applications. We have developed a deep neural network trained on a synthesized dataset that simulates potential geometric and photometric distortions encountered in a procam system illuminating a relatively smooth object (see Figure 1). Our trained network predicts the correspondence between projector and camera images, significantly improving registration accuracy across various procam configurations. By jointly considering the naturalness and feature richness of the projector images, our method minimizes visual disruptions in projected content without sacrificing precision. A user study confirms that our technique enhances perceived naturalness and usability compared to existing methods, validating its practical utility in real-world applications.

Related papers

• Robust Single-shot Structured Light 3D Imaging via Neural Feature Decoding [33.013553875034795]
  We consider the problem of active 3D imaging using single-shot structured light systems.<n>Traditional structured light methods typically decode depth correspondences through pixel-domain matching algorithms.<n>Inspired by recent advances in neural feature matching, we propose a learning-based structured light decoding framework.
  arXiv  Detail & Related papers  (2025-12-16T02:47:38Z)
• RePose-NeRF: Robust Radiance Fields for Mesh Reconstruction under Noisy Camera Poses [0.5328562411561214]
  We propose a robust framework that reconstructs high-quality, editable 3D meshes directly from multi-view images.<n>Our approach jointly refines camera poses while learning an implicit scene representation that captures fine geometric detail and appearance.
  arXiv  Detail & Related papers  (2025-11-11T18:25:58Z)
• DPCS: Path Tracing-Based Differentiable Projector-Camera Systems [49.69815958689441]
  Projector-camera systems (ProCams) simulation aims to model the physical project-and-capture process and associated scene parameters of a ProCams.<n>Recent advances use an end-to-end neural network to learn the project-and-capture process.<n>We introduce a novel path tracing-based differentiable projector-camera systems (DPCS), offering a differentiable ProCams simulation method.
  arXiv  Detail & Related papers  (2025-03-15T15:31:18Z)
• Neural Real-Time Recalibration for Infrared Multi-Camera Systems [2.249916681499244]
  There are no learning-free or neural techniques for real-time recalibration of infrared multi-camera systems. We propose a neural network-based method capable of dynamic real-time calibration.
  arXiv  Detail & Related papers  (2024-10-18T14:37:37Z)
• Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images [65.41966114373373]
  We present an improved solution to the neural image-based rendering problem in computer vision. The proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time.
  arXiv  Detail & Related papers  (2023-11-08T08:18:23Z)
• Neural Projection Mapping Using Reflectance Fields [11.74757574153076]
  We introduce a projector into a neural reflectance field that allows to calibrate the projector and photo realistic light editing. Our neural field consists of three neural networks, estimating geometry, material, and transmittance. We believe that neural projection mapping opens up the door to novel and exciting downstream tasks, through the joint optimization of the scene and projection images.
  arXiv  Detail & Related papers  (2023-06-11T05:33:10Z)
• How to Calibrate Your Event Camera [58.80418612800161]
  We propose a generic event camera calibration framework using image reconstruction. We show that neural-network-based image reconstruction is well suited for the task of intrinsic and extrinsic calibration of event cameras.
  arXiv  Detail & Related papers  (2021-05-26T07:06:58Z)
• Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo [61.6260594326246]
  We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
  arXiv  Detail & Related papers  (2021-03-22T18:06:58Z)
• Exploiting Raw Images for Real-Scene Super-Resolution [105.18021110372133]
  We study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. We also develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images.
  arXiv  Detail & Related papers  (2021-02-02T16:10:15Z)
• DeProCams: Simultaneous Relighting, Compensation and Shape Reconstruction for Projector-Camera Systems [91.45207885902786]
  We propose a novel end-to-end trainable model named DeProCams to learn the photometric and geometric mappings of ProCams. DeProCams explicitly decomposes the projector-camera image mappings into three subprocesses: shading attributes estimation, rough direct light estimation and photorealistic neural rendering. In our experiments, DeProCams shows clear advantages over previous arts with promising quality and being fully differentiable.
  arXiv  Detail & Related papers  (2020-03-06T05:49:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.