Configurable Learned Holography

• URL: http://arxiv.org/abs/2405.01558v2
• Date: Mon, 6 May 2024 09:02:58 GMT
• Title: Configurable Learned Holography
• Authors: Yicheng Zhan, Liang Shi, Wojciech Matusik, Qi Sun, Kaan Akşit,
• Abstract summary: We introduce a learned model that interactively computes 3D holograms from RGB-only 2D images for a variety of holographic displays. We enable our hologram computations to rely on identifying the correlation between depth estimation and 3D hologram synthesis tasks.
• Score: 33.45219677645646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the pursuit of advancing holographic display technology, we face a unique yet persistent roadblock: the inflexibility of learned holography in adapting to various hardware configurations. This is due to the variances in the complex optical components and system settings in existing holographic displays. Although the emerging learned approaches have enabled rapid and high-quality hologram generation, any alteration in display hardware still requires a retraining of the model. Our work introduces a configurable learned model that interactively computes 3D holograms from RGB-only 2D images for a variety of holographic displays. The model can be conditioned to predefined hardware parameters of existing holographic displays such as working wavelengths, pixel pitch, propagation distance, and peak brightness without having to retrain. In addition, our model accommodates various hologram types, including conventional single-color and emerging multi-color holograms that simultaneously use multiple color primaries in holographic displays. Notably, we enabled our hologram computations to rely on identifying the correlation between depth estimation and 3D hologram synthesis tasks within the learning domain for the first time in the literature. We employ knowledge distillation via a student-teacher learning strategy to streamline our model for interactive performance. Achieving up to a 2x speed improvement compared to state-of-the-art models while consistently generating high-quality 3D holograms with different hardware configurations.

Related papers

• Re-Thinking Inverse Graphics With Large Language Models [51.333105116400205]
  Inverse graphics -- inverting an image into physical variables that, when rendered, enable reproduction of the observed scene -- is a fundamental challenge in computer vision and graphics. We propose the Inverse-Graphics Large Language Model (IG-LLM), an inversegraphics framework centered around an LLM. We incorporate a frozen pre-trained visual encoder and a continuous numeric head to enable end-to-end training.
  arXiv  Detail & Related papers  (2024-04-23T16:59:02Z)
• Holo-VQVAE: VQ-VAE for phase-only holograms [1.534667887016089]
  Holography stands at the forefront of visual technology innovation, offering immersive, three-dimensional visualizations through the manipulation of light wave amplitude and phase. Modern research in hologram generation has predominantly focused on image-to-hologram conversion, producing holograms from existing images. We present Holo-VQVAE, a novel generative framework tailored for phase-only holograms (POHs)
  arXiv  Detail & Related papers  (2024-03-29T15:27:28Z)
• FLARE: Fast Learning of Animatable and Relightable Mesh Avatars [64.48254296523977]
  Our goal is to efficiently learn personalized animatable 3D head avatars from videos that are geometrically accurate, realistic, relightable, and compatible with current rendering systems. We introduce FLARE, a technique that enables the creation of animatable and relightable avatars from a single monocular video.
  arXiv  Detail & Related papers  (2023-10-26T16:13:00Z)
• AutoColor: Learned Light Power Control for Multi-Color Holograms [15.655689651318033]
  Multi-color holograms rely on simultaneous illumination from multiple light sources. We introduce AutoColor, the first learned method for estimating the optimal light source powers required for illuminating multi-color holograms.
  arXiv  Detail & Related papers  (2023-05-02T17:14:03Z)
• RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects [68.85305626324694]
  Ray-marching in Camera Space (RiCS) is a new method to represent the self-occlusions of foreground objects in 3D into a 2D self-occlusion map. We show that our representation map not only allows us to enhance the image quality but also to model temporally coherent complex shadow effects.
  arXiv  Detail & Related papers  (2022-05-14T05:35:35Z)
• Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators [44.73608798155336]
  Holographic near-eye displays offer unprecedented capabilities for virtual and augmented reality systems. We report advances in camera-calibrated wave propagation models for these types of holographic near-eye displays. Our framework is flexible in supporting runtime supervision with different types of content, including 2D and 2.5D RGBD images, 3D focal stacks, and 4D light fields.
  arXiv  Detail & Related papers  (2022-05-05T00:03:50Z)
• Learned holographic light transport [2.642698101441705]
  Holography algorithms often fall short in matching simulations with results from a physical holographic display. Our work addresses this mismatch by learning the holographic light transport in holographic displays. Our method can dramatically improve simulation accuracy and image quality in holographic displays.
  arXiv  Detail & Related papers  (2021-08-01T12:05:33Z)
• Pixel Codec Avatars [99.36561532588831]
  Pixel Codec Avatars (PiCA) is a deep generative model of 3D human faces. On a single Oculus Quest 2 mobile VR headset, 5 avatars are rendered in realtime in the same scene.
  arXiv  Detail & Related papers  (2021-04-09T23:17:36Z)
• Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition [67.9464567157846]
  We propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-06-29T12:53:58Z)
• Deep DIH : Statistically Inferred Reconstruction of Digital In-Line Holography by Deep Learning [1.4619386068190985]
  Digital in-line holography is commonly used to reconstruct 3D images from 2D holograms for microscopic objects. In this paper, we propose a novel implementation of autoencoder-based deep learning architecture for single-shot hologram reconstruction.
  arXiv  Detail & Related papers  (2020-04-25T20:39:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.