Self-supervised learning of hologram reconstruction using physics consistency

• URL: http://arxiv.org/abs/2209.08288v2
• Date: Sat, 17 Jun 2023 08:53:11 GMT
• Title: Self-supervised learning of hologram reconstruction using physics consistency
• Abstract summary: We report a self-supervised learning model, termed GedankenNet, that eliminates the need for labeled or experimental training data. After its self-supervised training, GedankenNet successfully generalized to experimental holograms of various unseen biological samples. Without access to experimental data or knowledge of real samples of interest or their spatial features, GedankenNet's self-supervised learning achieved complex-valued image reconstructions.
• Score: 0.30586855806896046
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The past decade has witnessed transformative applications of deep learning in various computational imaging, sensing and microscopy tasks. Due to the supervised learning schemes employed, these methods mostly depend on large-scale, diverse, and labeled training data. The acquisition and preparation of such training image datasets are often laborious and costly, also leading to biased estimation and limited generalization to new sample types. Here, we report a self-supervised learning model, termed GedankenNet, that eliminates the need for labeled or experimental training data, and demonstrate its effectiveness and superior generalization on hologram reconstruction tasks. Without prior knowledge about the sample types to be imaged, the self-supervised learning model was trained using a physics-consistency loss and artificial random images that are synthetically generated without any experiments or resemblance to real-world samples. After its self-supervised training, GedankenNet successfully generalized to experimental holograms of various unseen biological samples, reconstructing the phase and amplitude images of different types of objects using experimentally acquired test holograms. Without access to experimental data or knowledge of real samples of interest or their spatial features, GedankenNet's self-supervised learning achieved complex-valued image reconstructions that are consistent with the Maxwell's equations, and its output inference and object solutions accurately represent the wave propagation in free-space. GedankenNet framework also exhibits resilience to random, unknown perturbations in the physical forward model, including changes in the hologram distances, pixel size and illumination wavelength. This self-supervised learning of image reconstruction tasks creates new opportunities for various inverse problems in holography, microscopy and computational imaging fields.

Related papers

• Comparative Evaluation of 3D Reconstruction Methods for Immersive Visualization of Laboratory Objects [0.7241250079741012]
  We examined whether current 3D reconstruction methods can support the creation of realistic holographic representations of laboratory objects for educational use.<n>We compared four approaches: photogrammetry, a neural radiance field (NeRF)-based method, Gaussian splatting, and LiDAR.<n>Across objects, the NeRF-based method produced the most consistently high-fidelity representations.
  arXiv  Detail & Related papers  (2026-08-27T16:05:33Z)
• Generated Images Are Easier to Forget: A Machine Unlearning Perspective for Synthetic Image Detection [107.62647907393377]
  Large-scale vision models (LVMs) pre-trained on web-scale datasets exhibit exceptional generalization power.<n>We introduce two detection methods: 1) data-free detection, which prunes model parameters to induce unlearning without data access, and 2) data-driven detection, which optimize LVMs to unlearn knowledge tied to generated images.
  arXiv  Detail & Related papers  (2026-08-01T15:33:15Z)
• Self-supervised physics-informed generative networks for phase retrieval from a single X-ray hologram [0.4221292142376107]
  We present a self-learning approach for solving the inverse problem of phase retrieval in the near-field regime of Fresnel theory.<n>Unlike most deep learning approaches for phase retrieval, our approach does not require paired, unpaired, or simulated training data.
  arXiv  Detail & Related papers  (2025-08-21T13:06:06Z)
• Physics-Aware Style Transfer for Adaptive Holographic Reconstruction [1.8749305679160366]
  Inline holographic imaging presents an ill-posed inverse problem of reconstructing objects' complex amplitude from recorded diffraction patterns.<n>We present a physics-aware style transfer approach that interprets the object-to-sensor distance as an implicit style within diffraction patterns.<n>We show that the inverse mapping operation can be learned in an adaptive manner only with datasets composed of intensity measurements.
  arXiv  Detail & Related papers  (2025-07-01T06:56:51Z)
• Rewards-based image analysis in microscopy [2.906546126874626]
  Analyzing imaging and hyperspectral data is crucial across scientific fields, including biology, medicine, chemistry, and physics. Currently, this task relies on complex, human-designed iterative steps such as denoising, spatial sampling, keypoint detection, feature generation, clustering, dimensionality reduction, and physics-based deconvolutions. The introduction of machine learning over the past decade has accelerated tasks like image segmentation and object detection via supervised learning, and dimensionality reduction via unsupervised methods. Here, we discuss advances in reward-based, which adopt expert decision-making principles and demonstrate strong transfer learning across
  arXiv  Detail & Related papers  (2025-02-23T19:19:38Z)
• OAH-Net: A Deep Neural Network for Hologram Reconstruction of Off-axis Digital Holographic Microscope [5.835347176172883]
  We propose a novel reconstruction approach that integrates deep learning with the physical principles of off-axis holography. Our off-axis hologram network (OAH-Net) retrieves phase and amplitude images with errors that fall within the measurement error range attributable to hardware. This capability further expands off-axis holography's applications in both biological and medical studies.
  arXiv  Detail & Related papers  (2024-10-17T14:25:18Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Perceptual Artifacts Localization for Image Synthesis Tasks [59.638307505334076]
  We introduce a novel dataset comprising 10,168 generated images, each annotated with per-pixel perceptual artifact labels. A segmentation model, trained on our proposed dataset, effectively localizes artifacts across a range of tasks. We propose an innovative zoom-in inpainting pipeline that seamlessly rectifies perceptual artifacts in the generated images.
  arXiv  Detail & Related papers  (2023-10-09T10:22:08Z)
• A comparison of different atmospheric turbulence simulation methods for image restoration [64.24948495708337]
  Atmospheric turbulence deteriorates the quality of images captured by long-range imaging systems. Various deep learning-based atmospheric turbulence mitigation methods have been proposed in the literature. We systematically evaluate the effectiveness of various turbulence simulation methods on image restoration.
  arXiv  Detail & Related papers  (2022-04-19T16:21:36Z)
• The Devil is in the Frequency: Geminated Gestalt Autoencoder for Self-Supervised Visual Pre-Training [13.087987450384036]
  We present a new Masked Image Modeling (MIM), termed Geminated Autoencoder (Ge$2$-AE) for visual pre-training. Specifically, we equip our model with geminated decoders in charge of reconstructing image contents from both pixel and frequency space.
  arXiv  Detail & Related papers  (2022-04-18T09:22:55Z)
• Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types [50.1843146606122]
  A simple form of transfer learning is common in current state-of-the-art computer vision models. Previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood. In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains.
  arXiv  Detail & Related papers  (2021-03-24T16:24:20Z)
• SIR: Self-supervised Image Rectification via Seeing the Same Scene from Multiple Different Lenses [82.56853587380168]
  We propose a novel self-supervised image rectification (SIR) method based on an important insight that the rectified results of distorted images of the same scene from different lens should be the same. We leverage a differentiable warping module to generate the rectified images and re-distorted images from the distortion parameters. Our method achieves comparable or even better performance than the supervised baseline method and representative state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-11-30T08:23:25Z)
• Self-Supervised Linear Motion Deblurring [112.75317069916579]
  Deep convolutional neural networks are state-of-the-art for image deblurring. We present a differentiable reblur model for self-supervised motion deblurring. Our experiments demonstrate that self-supervised single image deblurring is really feasible.
  arXiv  Detail & Related papers  (2020-02-10T20:15:21Z)
• Learning Numerical Observers using Unsupervised Domain Adaptation [13.548174682737756]
  Medical imaging systems are commonly assessed by use of objective image quality measures. Supervised deep learning methods have been investigated to implement numerical observers for task-based image quality assessment. labeling large amounts of experimental data to train deep neural networks is tedious, expensive, and prone to subjective errors.
  arXiv  Detail & Related papers  (2020-02-03T22:58:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.