Learned Off-aperture Encoding for Wide Field-of-view RGBD Imaging

• URL: http://arxiv.org/abs/2507.22523v1
• Date: Wed, 30 Jul 2025 09:49:47 GMT
• Title: Learned Off-aperture Encoding for Wide Field-of-view RGBD Imaging
• Authors: Haoyu Wei, Xin Liu, Yuhui Liu, Qiang Fu, Wolfgang Heidrich, Edmund Y. Lam, Yifan Peng,
• Abstract summary: This work explores an additional design choice by positioning a DOE off-aperture, enabling a spatial unmixing of the degrees of freedom.<n> Experimental results reveal that the off-aperture DOE enhances the imaging quality by over 5 dB in PSNR at a FoV of approximately $45circ$ when paired with a simple thin lens.
• Score: 31.931929519577402
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: End-to-end (E2E) designed imaging systems integrate coded optical designs with decoding algorithms to enhance imaging fidelity for diverse visual tasks. However, existing E2E designs encounter significant challenges in maintaining high image fidelity at wide fields of view, due to high computational complexity, as well as difficulties in modeling off-axis wave propagation while accounting for off-axis aberrations. In particular, the common approach of placing the encoding element into the aperture or pupil plane results in only a global control of the wavefront. To overcome these limitations, this work explores an additional design choice by positioning a DOE off-aperture, enabling a spatial unmixing of the degrees of freedom and providing local control over the wavefront over the image plane. Our approach further leverages hybrid refractive-diffractive optical systems by linking differentiable ray and wave optics modeling, thereby optimizing depth imaging quality and demonstrating system versatility. Experimental results reveal that the off-aperture DOE enhances the imaging quality by over 5 dB in PSNR at a FoV of approximately $45^\circ$ when paired with a simple thin lens, outperforming traditional on-aperture systems. Furthermore, we successfully recover color and depth information at nearly $28^\circ$ FoV using off-aperture DOE configurations with compound optics. Physical prototypes for both applications validate the effectiveness and versatility of the proposed method.

Related papers

• Adaptive Stereo Depth Estimation with Multi-Spectral Images Across All Lighting Conditions [58.88917836512819]
  We propose a novel framework incorporating stereo depth estimation to enforce accurate geometric constraints. To mitigate the effects of poor lighting on stereo matching, we introduce Degradation Masking. Our method achieves state-of-the-art (SOTA) performance on the Multi-Spectral Stereo (MS2) dataset.
  arXiv  Detail & Related papers  (2024-11-06T03:30:46Z)
• Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems [15.976326291076377]
  We present Quasi-Global Search Optics (QGSO) to automatically design compound lens based computational imaging systems.<n>QGSO serves as a transformative end-to-end lens design paradigm for superior global search ability.
  arXiv  Detail & Related papers  (2024-04-30T01:59:25Z)
• Aperture Diffraction for Compact Snapshot Spectral Imaging [27.321750056840706]
  We demonstrate a compact, cost-effective snapshot spectral imaging system named Aperture Diffraction Imaging Spectrometer (ADIS) A new optical design that each point in the object space is multiplexed to discrete encoding locations on the mosaic filter sensor is introduced. The Cascade Shift-Shuffle Spectral Transformer (CSST) with strong perception of the diffraction degeneration is designed to solve a sparsity-constrained inverse problem.
  arXiv  Detail & Related papers  (2023-09-27T16:48:46Z)
• Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-free Multi-Exposure Image Fusion [60.221404321514086]
  Multi-exposure image fusion (MEF) has emerged as a prominent solution to address the limitations of digital imaging in representing varied exposure levels. This paper presents a Hybrid-Supervised Dual-Search approach for MEF, dubbed HSDS-MEF, which introduces a bi-level optimization search scheme for automatic design of both network structures and loss functions.
  arXiv  Detail & Related papers  (2023-09-03T08:07:26Z)
• Enhancing Low-light Light Field Images with A Deep Compensation Unfolding Network [52.77569396659629]
  This paper presents the deep compensation network unfolding (DCUNet) for restoring light field (LF) images captured under low-light conditions. The framework uses the intermediate enhanced result to estimate the illumination map, which is then employed in the unfolding process to produce a new enhanced result. To properly leverage the unique characteristics of LF images, this paper proposes a pseudo-explicit feature interaction module.
  arXiv  Detail & Related papers  (2023-08-10T07:53:06Z)
• Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising [62.347491141163225]
  4-D light field (LF) poses great challenges in achieving efficient and effective feature embedding. We propose a probabilistic-based feature embedding (PFE), which learns a feature embedding architecture by assembling various low-dimensional convolution patterns. Our experiments demonstrate the significant superiority of our methods on both real-world and synthetic 4-D LF images.
  arXiv  Detail & Related papers  (2023-06-15T03:46:40Z)
• D$^\text{2}$UF: Deep Coded Aperture Design and Unrolling Algorithm for Compressive Spectral Image Fusion [22.0246327137227]
  This paper presents the fusion of the compressive measurements of a low-spatial high-spectral resolution coded aperture snapshot spectral imager (CASSI) architecture and a high-spatial low-spectral resolution multispectral color filter array (MCFA) system. Unlike previous CSIF works, this paper proposes joint optimization of the sensing architectures and a reconstruction network in an end-to-end (E2E) manner.
  arXiv  Detail & Related papers  (2022-05-24T15:39:34Z)
• Universal and Flexible Optical Aberration Correction Using Deep-Prior Based Deconvolution [51.274657266928315]
  We propose a PSF aware plug-and-play deep network, which takes the aberrant image and PSF map as input and produces the latent high quality version via incorporating lens-specific deep priors. Specifically, we pre-train a base model from a set of diverse lenses and then adapt it to a given lens by quickly refining the parameters.
  arXiv  Detail & Related papers  (2021-04-07T12:00:38Z)
• Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics [72.9038524082252]
  We propose a compact single-shot monocular hyperspectral-depth (HS-D) imaging method. Our method uses a diffractive optical element (DOE), the point spread function of which changes with respect to both depth and spectrum. To facilitate learning the DOE, we present a first HS-D dataset by building a benchtop HS-D imager.
  arXiv  Detail & Related papers  (2020-09-01T14:19:35Z)
• Learning Wavefront Coding for Extended Depth of Field Imaging [4.199844472131922]
  Extended depth of field (EDoF) imaging is a challenging ill-posed problem. We propose a computational imaging approach for EDoF, where we employ wavefront coding via a diffractive optical element. We demonstrate results with minimal artifacts in various scenarios, including deep 3D scenes and broadband imaging.
  arXiv  Detail & Related papers  (2019-12-31T17:00:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.