Ptychoformer: A Physics-Guided Deep Learning Framework for Ptychographic Imaging

• URL: http://arxiv.org/abs/2412.06806v1
• Date: Mon, 25 Nov 2024 06:49:59 GMT
• Title: Ptychoformer: A Physics-Guided Deep Learning Framework for Ptychographic Imaging
• Authors: Han Yue, Jun Cheng, Yu-Xuan Ren, Philip Heng Wai Leong, Steve Feng Shu,
• Abstract summary: Ptychoformer is a physics-guided deep learning framework for ptychographic imaging. It aligns attention mechanisms and feature extraction with diffraction physics properties. It maintains robust performance under limited training data and low overlap ratios.
• Score: 9.387253806154098
• License:
• Abstract: Ptychographic imaging confronts limitations in applying deep learning (DL) for retrieval from diffraction patterns. Conventional neural architectures are optimized for natural images, overlooking the unique physical characteristics of diffraction data, including radial intensity decay and coherent information distributed in concentric rings. In this paper, we present Ptychoformer, a physics-guided DL framework for ptychographic imaging that aligns attention mechanisms and feature extraction with these diffraction physics properties through introducing a dual-branch architecture which accounts for both local and non-local dependencies from the patterns. It consists of a Polar Coordinate Attention (PCA) mechanism that is inspired by the Ewald construction in X-ray crystallography to enhance high-frequency component fidelity. Experimental results demonstrate Ptychoformer's superior performance across both simulated and real data in preserving fine details and suppressing artifacts. On simulated data, Ptychoformer achieves up to 5.4% higher PSNR and 4.2% higher SSIM for amplitude retrieval compared to existing methods. For real experimental data, it demonstrates up to 12.5% higher PSNR and 31.3% higher SSIM for amplitude retrieval. Notably, Ptychoformer maintains robust performance under limited training data and low overlap ratios, outperforming existing models.

Related papers

• Muographic Image Upsampling with Machine Learning for Built Infrastructure Applications [2.983520467199724]
  Muography, a non-invasive imaging technique, constructs three-dimensional density maps by detecting interactions of cosmic-ray muons. Cosmic-ray muons provide deep penetration and inherent safety due to their high momenta and natural source. However, the technology's reliance on this source results in constrained muon flux, leading to prolonged acquisition times. We developed a two-model deep learning approach to address these limitations.
  arXiv  Detail & Related papers  (2025-02-04T14:37:37Z)
• Understanding and Improving Training-Free AI-Generated Image Detections with Vision Foundation Models [68.90917438865078]
  Deepfake techniques for facial synthesis and editing pose serious risks for generative models. In this paper, we investigate how detection performance varies across model backbones, types, and datasets. We introduce Contrastive Blur, which enhances performance on facial images, and MINDER, which addresses noise type bias, balancing performance across domains.
  arXiv  Detail & Related papers  (2024-11-28T13:04:45Z)
• PASTA: Towards Flexible and Efficient HDR Imaging Via Progressively Aggregated Spatio-Temporal Alignment [91.38256332633544]
  PASTA is a Progressively Aggregated Spatio-Temporal Alignment framework for HDR deghosting. Our approach achieves effectiveness and efficiency by harnessing hierarchical representation during feature distanglement. Experimental results showcase PASTA's superiority over current SOTA methods in both visual quality and performance metrics.
  arXiv  Detail & Related papers  (2024-03-15T15:05:29Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization [56.95046107046027]
  We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for scene coordinate regression. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain.
  arXiv  Detail & Related papers  (2023-10-10T20:11:13Z)
• Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
  Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems. Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions. This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
  arXiv  Detail & Related papers  (2023-07-20T05:49:21Z)
• Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images [60.34381768479834]
  Recent advancements in diffusion models have enabled the generation of realistic deepfakes from textual prompts in natural language. We pioneer a systematic study on deepfake detection generated by state-of-the-art diffusion models.
  arXiv  Detail & Related papers  (2023-04-02T10:25:09Z)
• Deep Domain Adversarial Adaptation for Photon-efficient Imaging Based on Spatiotemporal Inception Network [11.58898808789911]
  In single-photon LiDAR, photon-efficient imaging captures the 3D structure of a scene by only several signal detected per pixel. Existing deep learning models for this task are trained on simulated datasets, which poses the domain shift challenge when applied to realistic scenarios. We propose a network (STIN) for photon-efficient imaging, which is able to precisely predict the depth from a sparse and high-noise photon counting histogram by fully exploiting spatial and temporal information.
  arXiv  Detail & Related papers  (2022-01-07T14:51:48Z)
• PAS-MEF: Multi-exposure image fusion based on principal component analysis, adaptive well-exposedness and saliency map [0.0]
  With regular low dynamic range (LDR) capture/display devices, significant details may not be preserved in images due to the huge dynamic range of natural scenes. This study proposes an efficient multi-exposure fusion (MEF) approach with a simple yet effective weight extraction method. Experimental comparisons with existing techniques demonstrate that the proposed method produces very strong statistical and visual results.
  arXiv  Detail & Related papers  (2021-05-25T10:22:43Z)
• Noise Reduction in X-ray Photon Correlation Spectroscopy with Convolutional Neural Networks Encoder-Decoder Models [0.0]
  We propose a computational approach for improving the signal-to-noise ratio in two-time correlation functions. CNN-ED models are based on Convolutional Neural Network-Decoder (CNN-ED) models. We demonstrate that the CNN-ED models trained on real-world experimental data help to effectively extract equilibrium dynamics parameters from two-time correlation functions.
  arXiv  Detail & Related papers  (2021-02-07T18:38:59Z)
• CNN-Based Image Reconstruction Method for Ultrafast Ultrasound Imaging [9.659642285903418]
  Ultrafast ultrasound (US) revolutionized biomedical imaging with its capability of acquiring full-view frames at over 1 kHz. It suffers from strong diffraction artifacts, mainly caused by grating lobes, side lobes, or edge waves. We propose a two-step convolutional neural network (CNN)-based image reconstruction method, compatible with real-time imaging.
  arXiv  Detail & Related papers  (2020-08-28T17:15:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.