A Real-world Display Inverse Rendering Dataset

• URL: http://arxiv.org/abs/2508.14411v1
• Date: Wed, 20 Aug 2025 04:15:19 GMT
• Title: A Real-world Display Inverse Rendering Dataset
• Authors: Seokjun Choi, Hoon-Gyu Chung, Yujin Jeon, Giljoo Nam, Seung-Hwan Baek,
• Abstract summary: Inverse rendering aims to reconstruct geometry and reflectance from captured images.<n>There is currently no public real-world dataset captured using display-camera systems.<n>We introduce the first real-world dataset for display-based inverse rendering.
• Score: 10.544409905036986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inverse rendering aims to reconstruct geometry and reflectance from captured images. Display-camera imaging systems offer unique advantages for this task: each pixel can easily function as a programmable point light source, and the polarized light emitted by LCD displays facilitates diffuse-specular separation. Despite these benefits, there is currently no public real-world dataset captured using display-camera systems, unlike other setups such as light stages. This absence hinders the development and evaluation of display-based inverse rendering methods. In this paper, we introduce the first real-world dataset for display-based inverse rendering. To achieve this, we construct and calibrate an imaging system comprising an LCD display and stereo polarization cameras. We then capture a diverse set of objects with diverse geometry and reflectance under one-light-at-a-time (OLAT) display patterns. We also provide high-quality ground-truth geometry. Our dataset enables the synthesis of captured images under arbitrary display patterns and different noise levels. Using this dataset, we evaluate the performance of existing photometric stereo and inverse rendering methods, and provide a simple, yet effective baseline for display inverse rendering, outperforming state-of-the-art inverse rendering methods. Code and dataset are available on our project page at https://michaelcsj.github.io/DIR/

Related papers

• OLATverse: A Large-scale Real-world Object Dataset with Precise Lighting Control [65.57356151743936]
  OLATverse is a large-scale dataset comprising around 9M images of 765 real-world objects.<n>Each object is captured using 35 DSLR cameras and 331 individually controlled light sources, enabling the simulation of diverse illumination conditions.<n>We construct an extensive evaluation set, establishing the first comprehensive real-world object-centric benchmark for inverse rendering and normal estimation.
  arXiv  Detail & Related papers  (2025-11-04T11:13:43Z)
• Inverse Image-Based Rendering for Light Field Generation from Single Images [30.856397422416517]
  We propose a novel view synthesis method for light field generation from only single images, named inverse image-based rendering.<n>Our method reconstructs light flows in a space from image pixels, which behaves in the opposite way to image-based rendering.<n>Our neural first stores the light flow of source rays from the input image, then computes the relationships among them through cross-attention.
  arXiv  Detail & Related papers  (2025-10-23T02:12:45Z)
• Reflect3r: Single-View 3D Stereo Reconstruction Aided by Mirror Reflections [55.248092751290834]
  Mirror reflections are common in everyday environments and can provide stereo information within a single capture.<n>We exploit this property by treating the reflection as an auxiliary view and designing a transformation that constructs a physically valid virtual camera.<n>This enables a multi-view stereo setup from a single image, simplifying the imaging process.
  arXiv  Detail & Related papers  (2025-09-24T23:00:22Z)
• Differentiable Mobile Display Photometric Stereo [8.436027244967438]
  Display photometric stereo uses a display as a programmable light source to illuminate a scene with diverse illumination conditions.<n>Recently, differentiable display photometric stereo demonstrated improved normal reconstruction accuracy by using learned display patterns.<n>We propose a more practical physics-based photometric stereo, differentiable mobile display photometric stereo.
  arXiv  Detail & Related papers  (2025-02-07T16:24:56Z)
• OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objects [56.065616159398324]
  We introduce OpenIllumination, a real-world dataset containing over 108K images of 64 objects with diverse materials. For each image in the dataset, we provide accurate camera parameters, illumination ground truth, and foreground segmentation masks.
  arXiv  Detail & Related papers  (2023-09-14T17:59:53Z)
• Differentiable Display Photometric Stereo [15.842538322034537]
  Photometric stereo leverages variations in illumination conditions to reconstruct surface normals. We present differentiable display photometric stereo (DDPS), addressing the design of display patterns. DDPS learns the display patterns that yield accurate normal reconstruction for a target system in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-06-23T07:05:08Z)
• WildLight: In-the-wild Inverse Rendering with a Flashlight [77.31815397135381]
  We propose a practical photometric solution for in-the-wild inverse rendering under unknown ambient lighting. Our system recovers scene geometry and reflectance using only multi-view images captured by a smartphone. We demonstrate by extensive experiments that our method is easy to implement, casual to set up, and consistently outperforms existing in-the-wild inverse rendering techniques.
  arXiv  Detail & Related papers  (2023-03-24T17:59:56Z)
• MAIR: Multi-view Attention Inverse Rendering with 3D Spatially-Varying Lighting Estimation [13.325800282424598]
  We propose a scene-level inverse rendering framework that uses multi-view images to decompose the scene into geometry, a SVBRDF, and 3D spatially-varying lighting. Our experiments show that the proposed method achieves better performance than single-view-based methods, but also achieves robust performance on unseen real-world scene.
  arXiv  Detail & Related papers  (2023-03-22T08:07:28Z)
• Enhancing Low-Light Images in Real World via Cross-Image Disentanglement [58.754943762945864]
  We propose a new low-light image enhancement dataset consisting of misaligned training images with real-world corruptions. Our model achieves state-of-the-art performances on both the newly proposed dataset and other popular low-light datasets.
  arXiv  Detail & Related papers  (2022-01-10T03:12:52Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)
• Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials [65.95928593628128]
  We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo technique. Our algorithm is suitable for perspective cameras and nearby point light sources.
  arXiv  Detail & Related papers  (2020-01-18T12:26:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.