From Pixels to Views: Learning Angular-Aware and Physics-Consistent Representations for Light Field Microscopy

• URL: http://arxiv.org/abs/2510.22577v1
• Date: Sun, 26 Oct 2025 08:28:05 GMT
• Title: From Pixels to Views: Learning Angular-Aware and Physics-Consistent Representations for Light Field Microscopy
• Authors: Feng He, Guodong Tan, Qiankun Li, Jun Yu, Quan Wen,
• Abstract summary: We introduce three key contributions to learning-based 3D reconstruction in light field microscopy.<n>First, we construct the XLFM-Zebrafish benchmark, a large-scale dataset and evaluation suite for XLFM reconstruction.<n>Second, we propose Masked View Modeling for Light Fields (MVN-LF), a self-supervised task that learns angular priors by predicting occluded views.<n>Third, we formulate the Optical Rendering Consistency Loss (ORC Loss), a differentiable rendering constraint that enforces alignment between predicted volumes and their PSF-based forward projections.
• Score: 12.900713570104749
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Light field microscopy (LFM) has become an emerging tool in neuroscience for large-scale neural imaging in vivo, notable for its single-exposure volumetric imaging, broad field of view, and high temporal resolution. However, learning-based 3D reconstruction in XLFM remains underdeveloped due to two core challenges: the absence of standardized datasets and the lack of methods that can efficiently model its angular-spatial structure while remaining physically grounded. We address these challenges by introducing three key contributions. First, we construct the XLFM-Zebrafish benchmark, a large-scale dataset and evaluation suite for XLFM reconstruction. Second, we propose Masked View Modeling for Light Fields (MVN-LF), a self-supervised task that learns angular priors by predicting occluded views, improving data efficiency. Third, we formulate the Optical Rendering Consistency Loss (ORC Loss), a differentiable rendering constraint that enforces alignment between predicted volumes and their PSF-based forward projections. On the XLFM-Zebrafish benchmark, our method improves PSNR by 7.7% over state-of-the-art baselines.

Related papers

• AnomalyVFM -- Transforming Vision Foundation Models into Zero-Shot Anomaly Detectors [6.6016630449883955]
  AnomalyVFM is a framework that turns any pretrained VFM into a strong zero-shot anomaly detector.<n>Our approach combines a robust three-stage synthetic dataset generation scheme with a parameter-efficient adaptation mechanism.<n>It achieves an average image-level AUROC of 94.1% across 9 diverse datasets, surpassing previous methods by significant 3.3 percentage points.
  arXiv  Detail & Related papers  (2026-01-28T12:02:58Z)
• One-Shot Refiner: Boosting Feed-forward Novel View Synthesis via One-Step Diffusion [57.824020826432815]
  We present a novel framework for high-fidelity novel view synthesis (NVS) from sparse images.<n>We design a Dual-Domain Detail Perception Module, which enables handling high-resolution images without being limited by the ViT backbone.<n>We develop a feature-guided diffusion network, which can preserve high-frequency details during the restoration process.
  arXiv  Detail & Related papers  (2026-01-20T17:11:55Z)
• Intern-GS: Vision Model Guided Sparse-View 3D Gaussian Splatting [95.61137026932062]
  Intern-GS is a novel approach to enhance the process of sparse-view Gaussian splatting.<n>We show that Intern-GS achieves state-of-the-art rendering quality across diverse datasets.
  arXiv  Detail & Related papers  (2025-05-27T05:17:49Z)
• DeepSparse: A Foundation Model for Sparse-View CBCT Reconstruction [9.579390210009521]
  Sparse-view reconstruction reduces radiation by using fewer X-ray projections while maintaining image quality.<n>Existing methods face challenges such as high computational demands and poor generalizability to different datasets.<n>We propose DeepSparse, the first foundation model for sparse-view CBCT reconstruction, featuring DiCE, a novel network that integrates multi-view 2D features and multi-scale 3D features.
  arXiv  Detail & Related papers  (2025-05-05T13:14:49Z)
• R-Meshfusion: Reinforcement Learning Powered Sparse-View Mesh Reconstruction with Diffusion Priors [6.102039677329314]
  Mesh reconstruction from multi-view images degrades significantly under sparse-view conditions.<n>Recent advances in diffusion models have demonstrated strong capabilities in synthesizing novel views from limited inputs.<n>We propose a novel framework that leverages diffusion models to enhance sparse-view mesh reconstruction in a principled and reliable manner.
  arXiv  Detail & Related papers  (2025-04-16T10:23:59Z)
• V2V3D: View-to-View Denoised 3D Reconstruction for Light-Field Microscopy [12.356249860549472]
  Light field microscopy (LFM) has gained significant attention due to its ability to capture snapshot-based, large-scale 3D fluorescence images.<n>Existing LFM reconstruction algorithms are highly sensitive to sensor noise or require hard-to-get ground-truth annotated data for training.<n>This paper introduces V2V3D, an unsupervised view2view-based framework that establishes a new paradigm for joint optimization of image denoising and 3D reconstruction.
  arXiv  Detail & Related papers  (2025-04-10T15:29:26Z)
• DiHuR: Diffusion-Guided Generalizable Human Reconstruction [51.31232435994026]
  We introduce DiHuR, a Diffusion-guided model for generalizable Human 3D Reconstruction and view synthesis from sparse, minimally overlapping images.<n>Our method integrates two key priors in a coherent manner: the prior from generalizable feed-forward models and the 2D diffusion prior, and it requires only multi-view image training, without 3D supervision.
  arXiv  Detail & Related papers  (2024-11-16T03:52:23Z)
• GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision [49.839374549646884]
  This paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception.<n>Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone.
  arXiv  Detail & Related papers  (2024-05-17T07:31:20Z)
• D-SCo: Dual-Stream Conditional Diffusion for Monocular Hand-Held Object Reconstruction [74.49121940466675]
  We introduce centroid-fixed dual-stream conditional diffusion for monocular hand-held object reconstruction. First, to avoid the object centroid from deviating, we utilize a novel hand-constrained centroid fixing paradigm. Second, we introduce a dual-stream denoiser to semantically and geometrically model hand-object interactions.
  arXiv  Detail & Related papers  (2023-11-23T20:14:50Z)
• Illumination and Shadows in Head Rotation: experiments with Denoising Diffusion Models [0.0]
  This study delves into the latent space of denoising diffusion models to identify compelling trajectories that can express continuous head rotation under varying lighting conditions.<n>A key contribution of our work is the generation of additional labels from the CelebA dataset,categorizing images into three groups based on prevalent illumination direction.
  arXiv  Detail & Related papers  (2023-08-11T10:14:22Z)
• Neural Poisson Surface Reconstruction: Resolution-Agnostic Shape Reconstruction from Point Clouds [53.02191521770926]
  We introduce Neural Poisson Surface Reconstruction (nPSR), an architecture for shape reconstruction that addresses the challenge of recovering 3D shapes from points. nPSR exhibits two main advantages: First, it enables efficient training on low-resolution data while achieving comparable performance at high-resolution evaluation. Overall, the neural Poisson surface reconstruction not only improves upon the limitations of classical deep neural networks in shape reconstruction but also achieves superior results in terms of reconstruction quality, running time, and resolution agnosticism.
  arXiv  Detail & Related papers  (2023-08-03T13:56:07Z)
• Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models [8.590026259176806]
  We propose a diffusion-model-based framework that overcomes the limitations of requiring reference data or prior knowledge about the degradation process. Our approach utilizes 2D diffusion models to consistently reconstruct 3D volumes and is well-suited for highly downsampled data.
  arXiv  Detail & Related papers  (2023-08-03T07:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.