ADOP: Approximate Differentiable One-Pixel Point Rendering
- URL: http://arxiv.org/abs/2110.06635v1
- Date: Wed, 13 Oct 2021 10:55:39 GMT
- Title: ADOP: Approximate Differentiable One-Pixel Point Rendering
- Authors: Darius R\"uckert, Linus Franke, Marc Stamminger
- Abstract summary: We present a point-based, differentiable neural rendering pipeline for scene refinement and novel view synthesis.
We show that our system is able to synthesize sharper and more consistent novel views than existing approaches.
- Score: 7.69748487650283
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We present a novel point-based, differentiable neural rendering pipeline for
scene refinement and novel view synthesis. The input are an initial estimate of
the point cloud and the camera parameters. The output are synthesized images
from arbitrary camera poses. The point cloud rendering is performed by a
differentiable renderer using multi-resolution one-pixel point rasterization.
Spatial gradients of the discrete rasterization are approximated by the novel
concept of ghost geometry. After rendering, the neural image pyramid is passed
through a deep neural network for shading calculations and hole-filling. A
differentiable, physically-based tonemapper then converts the intermediate
output to the target image. Since all stages of the pipeline are
differentiable, we optimize all of the scene's parameters i.e. camera model,
camera pose, point position, point color, environment map, rendering network
weights, vignetting, camera response function, per image exposure, and per
image white balance. We show that our system is able to synthesize sharper and
more consistent novel views than existing approaches because the initial
reconstruction is refined during training. The efficient one-pixel point
rasterization allows us to use arbitrary camera models and display scenes with
well over 100M points in real time.
Related papers
- Learning Robust Multi-Scale Representation for Neural Radiance Fields
from Unposed Images [65.41966114373373]
We present an improved solution to the neural image-based rendering problem in computer vision.
The proposed approach could synthesize a realistic image of the scene from a novel viewpoint at test time.
arXiv Detail & Related papers (2023-11-08T08:18:23Z) - $PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D
Reconstruction [97.06927852165464]
Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision.
We propose a novel method for single-image 3D reconstruction which generates a sparse point cloud via a conditional denoising diffusion process.
arXiv Detail & Related papers (2023-02-21T13:37:07Z) - View Synthesis with Sculpted Neural Points [64.40344086212279]
Implicit neural representations have achieved impressive visual quality but have drawbacks in computational efficiency.
We propose a new approach that performs view synthesis using point clouds.
It is the first point-based method to achieve better visual quality than NeRF while being more than 100x faster in rendering speed.
arXiv Detail & Related papers (2022-05-12T03:54:35Z) - ERF: Explicit Radiance Field Reconstruction From Scratch [12.254150867994163]
We propose a novel explicit dense 3D reconstruction approach that processes a set of images of a scene with sensor poses and calibrations and estimates a photo-real digital model.
One of the key innovations is that the underlying volumetric representation is completely explicit.
We show that our method is general and practical. It does not require a highly controlled lab setup for capturing, but allows for reconstructing scenes with a vast variety of objects.
arXiv Detail & Related papers (2022-02-28T19:37:12Z) - PICCOLO: Point Cloud-Centric Omnidirectional Localization [20.567452635590943]
We present PICCOLO, a simple and efficient algorithm for omnidirectional localization.
Our pipeline works in an off-the-shelf manner with a single image given as a query.
PICCOLO outperforms existing omnidirectional localization algorithms in both accuracy and stability when evaluated in various environments.
arXiv Detail & Related papers (2021-08-14T14:19:37Z) - DeepI2P: Image-to-Point Cloud Registration via Deep Classification [71.3121124994105]
DeepI2P is a novel approach for cross-modality registration between an image and a point cloud.
Our method estimates the relative rigid transformation between the coordinate frames of the camera and Lidar.
We circumvent the difficulty by converting the registration problem into a classification and inverse camera projection optimization problem.
arXiv Detail & Related papers (2021-04-08T04:27:32Z) - Back to the Feature: Learning Robust Camera Localization from Pixels to
Pose [114.89389528198738]
We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model.
The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching.
arXiv Detail & Related papers (2021-03-16T17:40:12Z) - TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer [6.320273914694594]
We propose and evaluate a neural point-based graphics method that can model semi-transparent scene parts.
We show that novel views of semi-transparent point cloud scenes can be generated after training with our approach.
arXiv Detail & Related papers (2020-09-06T21:19:18Z) - Neural Geometric Parser for Single Image Camera Calibration [17.393543270903653]
We propose a neural geometric learning single image camera calibration for man-made scenes.
Our approach considers both semantic and geometric cues, resulting in significant accuracy improvement.
The experimental results reveal that the performance of our neural approach is significantly higher than that of existing state-of-the-art camera calibration techniques.
arXiv Detail & Related papers (2020-07-23T08:29:00Z) - NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis [78.5281048849446]
We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes.
Our algorithm represents a scene using a fully-connected (non-convolutional) deep network.
Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses.
arXiv Detail & Related papers (2020-03-19T17:57:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.