Gaussian Process Prior Variational Autoencoder for Endoscopic Videos

• URL: http://arxiv.org/abs/2606.19908v1
• Date: Thu, 18 Jun 2026 08:03:55 GMT
• Title: Gaussian Process Prior Variational Autoencoder for Endoscopic Videos
• Abstract summary: Video sequences are routinely degraded by specular reflections, motion artifacts, and missing frames.<n>We introduce a Gaussian Process Prior Variational Autoencoder (GPVAE) framework for endoscopic video restoration.<n>On the C3VDv2 colonoscopy dataset, the best GPVAE variants reduced image reconstruction RMSE by 21.9% on average.
• Score: 6.831828660437534
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Endoscopic video analysis is essential for gastrointestinal diagnosis and computer-assisted interventions, but video sequences are routinely degraded by specular reflections, motion artifacts, and missing frames. These transient corruptions can distract clinicians, reduce image interpretability, and disrupt downstream tasks such as 3D reconstruction and navigation. Effective restoration therefore requires methods that exploit temporal continuity rather than treating frames in isolation. We introduce a Gaussian Process Prior Variational Autoencoder (GPVAE) framework for endoscopic video restoration that replaces the standard factorized latent prior with a temporal Gaussian process prior, enabling interpolation of missing frames with uncertainty-aware reconstruction. The framework combines endoscopy-specific encoders, including a convolutional EndoVAE backbone and pretrained Vision Transformer encoders from GastroNet-5M, with two scalable GP approximations: Hierarchical Prior Approximation (HPA) and Sparse Precision Approximation (SPA). Specular reflections are handled using a DUCKNet-based masking pipeline that excludes corrupted pixels from the reconstruction objective. On the C3VDv2 colonoscopy dataset, the best GPVAE variants reduced image reconstruction RMSE by 21.9\% on average, and by up to 26.1\%, relative to matched VAE baselines. Downstream trajectory RMSE was reduced by 12.7\% on average across classical visual odometry and a pretrained PoseNet, at an average increase of 27.3\% in training time per epoch. Finally, the GP posterior provides per-frame uncertainty estimates that reflect temporal support and offer a confidence signal for restored frames.

Related papers

• tcnerv:dual-domain temporal context modeling for implicit neural video compression [6.025390826523129]
  TCNeRV exploits reconstructed context in both feature and embedding domains.<n>It achieves an average PSNR of 36.08 dB on the UVG dataset, outperforming HNeRV-Boost by 2.20 dB.
  arXiv  Detail & Related papers  (2026-09-15T09:00:18Z)
• LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics [50.45297537821907]
  We introduce LeVJEPA, the first video encoder trained under LeJEPA's collapse-free objective.<n>A single encoder is trained with an invariance loss over global and local views of a clip.<n>LeVJEPA matches or surpasses V-JEPA 2 across ViT-S/B/L at 5.6 to 20.8x less pretraining compute.
  arXiv  Detail & Related papers  (2026-08-27T17:26:24Z)
• A Zero-Shot Deep Image Prior Framework for Denoising and Deconvolution in Fluorescence Microscopy [6.129338407421482]
  We propose SDIP, a zero-shot deep image prior framework that sequentially performs denoising and deconvolution without external training data.<n>Experiments on the BioSR dataset across multiple cellular structures demonstrate that SDIP improves both signal-to-noise ratio and resolution.<n>The proposed framework may also provide useful insights for designing physically guided DIP methods for other inverse problems.
  arXiv  Detail & Related papers  (2026-06-26T01:52:27Z)
• Uncertainty-quantified Pulse Signal Recovery from Facial Video using Regularized Stochastic Interpolants [32.782480100477976]
  Photoplethysmography (i) is an optical procedure which recovers a human's blood volume pulse (BVP) waveform using pixel readout from a camera.<n>Current algorithms to solve the i task have shown outstanding performance benchmark datasets.<n>No state-of-the art algorithms, vectors to the best of our knowledge, perform test-time sampling of solution space.
  arXiv  Detail & Related papers  (2026-04-12T19:00:49Z)
• Exploiting Intermediate Reconstructions in Optical Coherence Tomography for Test-Time Adaption of Medical Image Segmentation [3.7585770539752104]
  Primary health care frequently relies on low-cost imaging devices, which are commonly used for screening purposes.<n>Such algorithms typically employ iterative reconstruction methods that incorporate domain-specific prior knowledge.<n>We propose IRTTA to exploit these intermediate representations at test-time by adapting the normalization-layer parameters of a frozen downstream network.
  arXiv  Detail & Related papers  (2026-03-05T10:48:37Z)
• SFP: Real-World Scene Recovery Using Spatial and Frequency Priors [84.27251794411673]
  Scene recovery serves as a critical task for various computer vision applications.<n>We propose Spatial and Frequency Priors (SFP) for real-world scene recovery.
  arXiv  Detail & Related papers  (2025-12-09T05:24:25Z)
• BronchOpt : Vision-Based Pose Optimization with Fine-Tuned Foundation Models for Accurate Bronchoscopy Navigation [6.915058920280426]
  We propose a vision-based pose optimization framework for 2D-3D registration between intra-operative endoscopic views and pre-operative CT anatomy.<n>A fine-tuned modality- and domain-invariant encoder enables direct similarity between real endoscopic RGB frames and CT-rendered depth maps.<n>Our model achieves an average translational error of 2.65 mm and a rotational error of 0.19 rad, demonstrating accurate and stable localization.
  arXiv  Detail & Related papers  (2025-11-12T15:58:05Z)
• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• TPA: Temporal Prompt Alignment for Fetal Congenital Heart Defect Classification [2.3974223785103166]
  Congenital heart defect (CHD) detection in ultrasound videos is hindered by image noise and probe positioning variability.<n>We propose Temporal Prompt Alignment (TPA), a method leveraging foundation image-text model and prompt-aware contrastive learning.<n>TPA extracts features from each frame of video subclips using an image encoder, aggregates them with a trainable temporal extractor, and aligns the video representation with class-specific text prompts.
  arXiv  Detail & Related papers  (2025-08-21T06:36:47Z)
• Focus on Texture: Rethinking Pre-training in Masked Autoencoders for Medical Image Classification [6.641920678512381]
  Masked Autoencoders (MAEs) have emerged as a dominant strategy for self-supervised representation learning in natural images.<n>We propose a novel MAE based pre-training framework, GLCM-MAE, using reconstruction loss based on matching GLCM.<n>GLCM-MAE outperforms the current state-of-the-art across four tasks.
  arXiv  Detail & Related papers  (2025-07-15T00:12:26Z)
• APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers [71.2294205496784]
  We propose textbfAPHQ-ViT, a novel PTQ approach based on importance estimation with Average Perturbation Hessian (APH)<n>We show that APHQ-ViT using linear quantizers outperforms existing PTQ methods by substantial margins in 3-bit and 4-bit across different vision tasks.
  arXiv  Detail & Related papers  (2025-04-03T11:48:56Z)
• Pathology Image Compression with Pre-trained Autoencoders [52.208181380986524]
  Whole Slide Images in digital histopathology pose significant storage, transmission, and computational efficiency challenges.<n>Standard compression methods, such as JPEG, reduce file sizes but fail to preserve fine-grained phenotypic details critical for downstream tasks.<n>In this work, we repurpose autoencoders (AEs) designed for Latent Diffusion Models as an efficient learned compression framework for pathology images.
  arXiv  Detail & Related papers  (2025-03-14T17:01:17Z)
• Making Reconstruction-based Method Great Again for Video Anomaly Detection [64.19326819088563]
  Anomaly detection in videos is a significant yet challenging problem. Existing reconstruction-based methods rely on old-fashioned convolutional autoencoders. We propose a new autoencoder model for enhanced consecutive frame reconstruction.
  arXiv  Detail & Related papers  (2023-01-28T01:57:57Z)
• Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging [142.11622043078867]
  We propose a principled Degradation-Aware Unfolding Framework (DAUF) that estimates parameters from the compressed image and physical mask, and then uses these parameters to control each iteration. By plugging HST into DAUF, we establish the first Transformer-based deep unfolding method, Degradation-Aware Unfolding Half-Shuffle Transformer (DAUHST) for HSI reconstruction.
  arXiv  Detail & Related papers  (2022-05-20T11:37:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.