GenRec: Knowing Where to Reconstruct and Where to Generate

• URL: http://arxiv.org/abs/2608.17832v1
• Date: Tue, 18 Aug 2026 14:31:19 GMT
• Title: GenRec: Knowing Where to Reconstruct and Where to Generate
• Abstract summary: GenRec is a flow matching model that builds the reconstruction--generation split directly into its architecture, supervision, and gradient flow.<n>It attains the best reconstruction fidelity in observed regions while also surpassing purely generative baselines on perceptual quality in unobserved ones.
• Score: 106.48347271421765
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Generative novel view synthesis from sparse input images is rarely all reconstruction or all generation: pixels visible in some source view have a unique correct value modulated only by view-dependent shading, while pixels in disocclusions or beyond the captured volume admit a distribution of plausible completions. Existing generative novel-view-synthesis methods conflate these regimes under a single uniform loss, blurring the line between geometric fidelity and creative hallucinations even when scene geometry is injected through warped point clouds or projected depth. We introduce GenRec, a multi-view flow matching model that builds the reconstruction--generation split directly into its architecture, supervision, and gradient flow. Guided by an observation mask derived from the source cameras and a monocular depth estimator, a flow matching backbone jointly denoises RGB and scene-coordinate maps across all target views, while a pixel-space refinement stage restores high-frequency detail on observed pixels; the same mask gates supervision so regression signals do not contaminate the generative prior. Across RealEstate10K, DL3DV-10K, and Mip-NeRF~360, in both single-view extrapolation and two-view interpolation, GenRec attains the best reconstruction fidelity in observed regions while also surpassing purely generative baselines on perceptual quality in unobserved ones, showing the effectiveness of our approach.

Related papers

• Filling the Unseen: Scene Extrapolation via 3D Gaussian Splatting [10.582831709592789]
  3D Gaussian Splatting reconstruction within training view distribution degrades on out-of-distribution novel views, exhibiting holes in unobserved regions and artifacts in observable areas.<n>Recent works formulate this task as extrapolation and conditioning and try to address it with generative models, but remain limited in extrapolation scale and quality.<n>We propose a holistic framework for extrapolation and conditioning, with an independent camera view detection mechanism to enable parallel conflict-free extrapolation.
  arXiv  Detail & Related papers  (2026-09-06T19:02:27Z)
• SplatGuide: Geometric Priors from 3D Gaussians for Pose-Free Novel View Synthesis [42.23526579547921]
  SplatGuide reuses a single 3DGS scene across three complementary roles.<n>Per-Gaussian source-view indices are rendered into a target-view voting map.<n>Reconstruction tokens supply feature-level guidance via cross-attention.
  arXiv  Detail & Related papers  (2026-08-17T17:45:57Z)
• VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors [79.88300861391114]
  VidSplat is a training-free generative reconstruction framework.<n>We tackle two key challenges that enable the effective integration of generation and reconstruction.<n>Experiments on widely used benchmarks demonstrate our superior performance in sparse-view scene reconstruction.
  arXiv  Detail & Related papers  (2026-05-12T02:20:31Z)
• Pseudo-View Enhancement via Confidence Fusion for Unposed Sparse-View Reconstruction [7.8268279161630785]
  3D scene reconstruction under unposed sparse viewpoints is a highly challenging yet practically important problem.<n>We propose a novel framework for sparse-view outdoor reconstruction that achieves high-quality results through bidirectional pseudo frame restoration.<n>Designs significantly enhance reconstruction completeness, suppress floating artifacts and improve overall geometric consistency under extreme view sparsity.
  arXiv  Detail & Related papers  (2026-02-25T03:45:47Z)
• InpaintHuman: Reconstructing Occluded Humans with Multi-Scale UV Mapping and Identity-Preserving Diffusion Inpainting [64.42884719282323]
  InpaintHuman is a novel method for generating high-fidelity, complete, and animatable avatars from occluded monocular videos.<n>Our approach employs direct pixel-level supervision to ensure identity fidelity.
  arXiv  Detail & Related papers  (2026-01-05T13:26:02Z)
• G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior [53.762256749551284]
  We identify accurate geometry as the fundamental prerequisite for effectively exploiting generative models to enhance 3D scene reconstruction.<n>We incorporate this geometry guidance throughout the generative pipeline to improve visibility mask estimation, guide novel view selection, and enhance multi-view consistency when inpainting with video diffusion models.<n>Our method naturally supports single-view inputs and unposed videos, with strong generalizability in both indoor and outdoor scenarios.
  arXiv  Detail & Related papers  (2025-10-14T03:06:28Z)
• OracleGS: Grounding Generative Priors for Sparse-View Gaussian Splatting [78.70702961852119]
  OracleGS reconciles generative completeness with regressive fidelity for sparse view Gaussian Splatting.<n>Our approach conditions the powerful generative prior on multi-view geometric evidence, filtering hallucinatory artifacts while preserving plausible completions in under-constrained regions.
  arXiv  Detail & Related papers  (2025-09-27T11:19:32Z)
• DeOcc-1-to-3: 3D De-Occlusion from a Single Image via Self-Supervised Multi-View Diffusion [50.90541069907167]
  We propose DeOcc-1-to-3, an end-to-end framework for occlusion-aware multi-view generation.<n>Our self-supervised training pipeline leverages occluded-unoccluded image pairs and pseudo-ground-truth views to teach the model structure-aware completion and view consistency.
  arXiv  Detail & Related papers  (2025-06-26T17:58:26Z)
• GenFusion: Closing the Loop between Reconstruction and Generation via Videos [24.195304481751602]
  We propose a reconstruction-driven video diffusion model that learns to condition video frames on artifact-prone RGB-D renderings.<n>We also propose a cyclical fusion pipeline that iteratively adds restoration frames from the generative model to the training set.
  arXiv  Detail & Related papers  (2025-03-27T07:16:24Z)
• GenS: Generalizable Neural Surface Reconstruction from Multi-View Images [20.184657468900852]
  GenS is an end-to-end generalizable neural surface reconstruction model. Our representation is more powerful, which can recover high-frequency details while maintaining global smoothness. Experiments on popular benchmarks show that our model can generalize well to new scenes.
  arXiv  Detail & Related papers  (2024-06-04T17:13:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.