End-to-end differentiable design of geometric waveguide displays

• URL: http://arxiv.org/abs/2601.04370v1
• Date: Wed, 07 Jan 2026 20:19:11 GMT
• Title: End-to-end differentiable design of geometric waveguide displays
• Authors: Xinge Yang, Zhaocheng Liu, Zhaoyu Nie, Qingyuan Fan, Zhimin Shi, Jim Bonar, Wolfgang Heidrich,
• Abstract summary: We present the first end-to-end differentiable optimization framework for geometric waveguides.<n>A differentiable Monte Carlo ray tracer avoids the exponential growth of deterministic ray splitting.<n>We also jointly optimize the waveguide and an image preprocessing network to improve perceived image quality.
• Score: 8.222618391106979
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Geometric waveguides are a promising architecture for optical see-through augmented reality displays, but their performance is severely bottlenecked by the difficulty of jointly optimizing non-sequential light transport and polarization-dependent multilayer thin-film coatings. Here we present the first end-to-end differentiable optimization framework for geometric waveguide that couples non-sequential Monte Carlo polarization ray tracing with a differentiable transfer-matrix thin-film solver. A differentiable Monte Carlo ray tracer avoids the exponential growth of deterministic ray splitting while enabling gradients backpropagation from eyebox metrics to design parameters. With memory-saving strategies, we optimize more than one thousand layer-thickness parameters and billions of non-sequential ray-surface intersections on a single multi-GPU workstation. Automated layer pruning is achieved by starting from over-parameterized stacks and driving redundant layers to zero thickness under discrete manufacturability constraints, effectively performing topology optimization to discover optimal coating structures. On a representative design, starting from random initialization within thickness bounds, our method increases light efficiency from 4.1\% to 33.5\% and improves eyebox and FoV uniformity by $\sim$17$\times$ and $\sim$11$\times$, respectively. Furthermore, we jointly optimize the waveguide and an image preprocessing network to improve perceived image quality. Our framework not only enables system-level, high-dimensional coating optimization inside the waveguide, but also expands the scope of differentiable optics for next-generation optical design.

Related papers

• BONNI: Gradient-Informed Bayesian and Interior Point Optimization for Efficient Inverse Design in Nanophotonics [28.968853022624444]
  Inverse design provides a systematic approach for developing high-performance nanophotonic devices.<n>We introduce BONNI: Bayesian optimization through neural network ensemble surrogates with interior point optimization.<n>We demonstrate BONNI's capabilities in the design of a distributed Bragg reflector and a dual-layer grating coupler.
  arXiv  Detail & Related papers  (2026-02-20T11:26:45Z)
• Bridging Geometry-Coherent Text-to-3D Generation with Multi-View Diffusion Priors and Gaussian Splatting [51.08718483081347]
  We propose a framework that couples multi-view joint distribution priors to ensure geometrically consistent 3D generation.<n>We derive an effective optimization rule that effectively couples multi-view priors to guide optimization across different viewpoints.<n>We employ a deformable tetrahedral grid, from 3D-GS and refined through CSD, to produce high-quality, refined meshes.
  arXiv  Detail & Related papers  (2025-05-07T09:12:45Z)
• Successive optimization of optics and post-processing with differentiable coherent PSF operator and field information [9.527960631238173]
  We introduce a precise optical simulation model, and every operation in pipeline is differentiable.<n>To efficiently address various degradation, we design a joint optimization procedure that leverages field information.
  arXiv  Detail & Related papers  (2024-12-19T07:49:40Z)
• A Differentiable Wave Optics Model for End-to-End Computational Imaging System Optimization [19.83939112821776]
  End-to-end optimization has emerged as a powerful data-driven method for computational imaging system design.<n>It is challenging to model both aberration and diffraction in light transport for end-to-end optimization of compound optics.<n>We propose a differentiable optics simulator that efficiently models both aberration and diffraction for compound optics.
  arXiv  Detail & Related papers  (2024-12-13T00:57:47Z)
• $ν$-DBA: Neural Implicit Dense Bundle Adjustment Enables Image-Only Driving Scene Reconstruction [31.64067619807023]
  $nu$-DBA implements geometric dense bundle adjustment (DBA) using 3D neural implicit surfaces for map parametrization. We fine-tune the optical flow model with per-scene self-supervision to further improve the quality of the dense mapping.
  arXiv  Detail & Related papers  (2024-04-29T05:29:26Z)
• Flexible Isosurface Extraction for Gradient-Based Mesh Optimization [65.76362454554754]
  This work considers gradient-based mesh optimization, where we iteratively optimize for a 3D surface mesh by representing it as the isosurface of a scalar field. We introduce FlexiCubes, an isosurface representation specifically designed for optimizing an unknown mesh with respect to geometric, visual, or even physical objectives.
  arXiv  Detail & Related papers  (2023-08-10T06:40:19Z)
• TMM-Fast: A Transfer Matrix Computation Package for Multilayer Thin-Film Optimization [62.997667081978825]
  An advanced thin-film structure can consist of multiple materials with different thicknesses and numerous layers. Design and optimization of complex thin-film structures with multiple variables is a computationally heavy problem that is still under active research. We propose the Python package TMM-Fast which enables parallelized computation of reflection and transmission of light at different angles of incidence and wavelengths through the multilayer thin-film.
  arXiv  Detail & Related papers  (2021-11-24T14:47:37Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)
• Differentiable Rendering with Perturbed Optimizers [85.66675707599782]
  Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision. Our work highlights the link between some well-known differentiable formulations and randomly smoothed renderings. We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction.
  arXiv  Detail & Related papers  (2021-10-18T08:56:23Z)
• Progressive Encoding for Neural Optimization [92.55503085245304]
  We show the competence of the PPE layer for mesh transfer and its advantages compared to contemporary surface mapping techniques. Most importantly, our technique is a parameterization-free method, and thus applicable to a variety of target shape representations.
  arXiv  Detail & Related papers  (2021-04-19T08:22:55Z)
• Parameterized Reinforcement Learning for Optical System Optimization [0.0]
  A multi-layer optical system with designated optical characteristics is designed by several discrete and continuous parameters. Most methods merely determine the optimal thicknesses of the system's layers. We propose a method that considers the stacking of consecutive layers as parameterized actions in a Markov decision process.
  arXiv  Detail & Related papers  (2020-10-09T10:53:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.