Learning to Enhance Aperture Phasor Field for Non-Line-of-Sight Imaging

• URL: http://arxiv.org/abs/2407.18574v2
• Date: Mon, 29 Jul 2024 03:33:47 GMT
• Title: Learning to Enhance Aperture Phasor Field for Non-Line-of-Sight Imaging
• Authors: In Cho, Hyunbo Shim, Seon Joo Kim,
• Abstract summary: This paper aims to facilitate more practical NLOS imaging by reducing the number of samplings and scan areas. We leverage a denoising autoencoder scheme to acquire rich and noise-robust representations in the measurement space. We introduce a phasor-based pipeline designed to limit the spectrum of our network to the frequency range of interests.
• Score: 22.365437882740657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper aims to facilitate more practical NLOS imaging by reducing the number of samplings and scan areas. To this end, we introduce a phasor-based enhancement network that is capable of predicting clean and full measurements from noisy partial observations. We leverage a denoising autoencoder scheme to acquire rich and noise-robust representations in the measurement space. Through this pipeline, our enhancement network is trained to accurately reconstruct complete measurements from their corrupted and partial counterparts. However, we observe that the \naive application of denoising often yields degraded and over-smoothed results, caused by unnecessary and spurious frequency signals present in measurements. To address this issue, we introduce a phasor-based pipeline designed to limit the spectrum of our network to the frequency range of interests, where the majority of informative signals are detected. The phasor wavefronts at the aperture, which are band-limited signals, are employed as inputs and outputs of the network, guiding our network to learn from the frequency range of interests and discard unnecessary information. The experimental results in more practical acquisition scenarios demonstrate that we can look around the corners with $16\times$ or $64\times$ fewer samplings and $4\times$ smaller apertures. Our code is available at https://github.com/join16/LEAP.

Related papers

• TBSN: Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising [94.09442506816724]
  Blind-spot networks (BSN) have been prevalent network architectures in self-supervised image denoising (SSID) We present a transformer-based blind-spot network (TBSN) by analyzing and redesigning the transformer operators that meet the blind-spot requirement. For spatial self-attention, an elaborate mask is applied to the attention matrix to restrict its receptive field, thus mimicking the dilated convolution. For channel self-attention, we observe that it may leak the blind-spot information when the channel number is greater than spatial size in the deep layers of multi-scale architectures.
  arXiv  Detail & Related papers  (2024-04-11T15:39:10Z)
• Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning [81.98675881423131]
  This research addresses the challenge of developing a universal deepfake detector that can effectively identify unseen deepfake images. Existing frequency-based paradigms have relied on frequency-level artifacts introduced during the up-sampling in GAN pipelines to detect forgeries. We introduce a novel frequency-aware approach called FreqNet, centered around frequency domain learning, specifically designed to enhance the generalizability of deepfake detectors.
  arXiv  Detail & Related papers  (2024-03-12T01:28:00Z)
• Self-supervised Image Denoising with Downsampled Invariance Loss and Conditional Blind-Spot Network [12.478287906337194]
  Most representative self-supervised denoisers are based on blind-spot networks. A standard blind-spot network fails to reduce real camera noise due to the pixel-wise correlation of noise. We propose a novel self-supervised training framework that can remove real noise.
  arXiv  Detail & Related papers  (2023-04-19T08:55:27Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• FMNet: Latent Feature-wise Mapping Network for Cleaning up Noisy Micro-Doppler Spectrogram [2.9849405664643585]
  noisy surroundings cause uninterpretable motion patterns on the micro-Doppler spectrogram. radar returns often suffer from multipath, clutter and interference. We propose a latent feature-wise mapping strategy, called Feature Mapping Network (FMNet), to transform measured spectrograms.
  arXiv  Detail & Related papers  (2021-07-09T19:20:41Z)
• SignalNet: A Low Resolution Sinusoid Decomposition and Estimation Network [79.04274563889548]
  We propose SignalNet, a neural network architecture that detects the number of sinusoids and estimates their parameters from quantized in-phase and quadrature samples. We introduce a worst-case learning threshold for comparing the results of our network relative to the underlying data distributions. In simulation, we find that our algorithm is always able to surpass the threshold for three-bit data but often cannot exceed the threshold for one-bit data.
  arXiv  Detail & Related papers  (2021-06-10T04:21:20Z)
• Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain [88.7339322596758]
  We present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery. SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
  arXiv  Detail & Related papers  (2021-03-02T16:45:08Z)
• Robust and fast post-processing of single-shot spin qubit detection events with a neural network [0.0]
  We train neural networks to classify a collection of single-shot spin detection events. We find an increase of 7 % in the visibility of the Rabi-oscillation when we employ a network trained by synthetic readout traces.
  arXiv  Detail & Related papers  (2020-12-08T19:13:09Z)
• Differentiable Manifold Reconstruction for Point Cloud Denoising [23.33652755967715]
  3D point clouds are often perturbed by noise due to the inherent limitation of acquisition equipments. We propose to learn the underlying manifold of a noisy point cloud from differentiably subsampled points. We show that our method significantly outperforms state-of-the-art denoising methods under both synthetic noise and real world noise.
  arXiv  Detail & Related papers  (2020-07-27T13:31:41Z)
• ADRN: Attention-based Deep Residual Network for Hyperspectral Image Denoising [52.01041506447195]
  We propose an attention-based deep residual network to learn a mapping from noisy HSI to the clean one. Experimental results demonstrate that our proposed ADRN scheme outperforms the state-of-the-art methods both in quantitative and visual evaluations.
  arXiv  Detail & Related papers  (2020-03-04T08:36:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.