Modular Primitives for High-Performance Differentiable Rendering

• URL: http://arxiv.org/abs/2011.03277v1
• Date: Fri, 6 Nov 2020 10:48:43 GMT
• Title: Modular Primitives for High-Performance Differentiable Rendering
• Authors: Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, Timo Aila
• Abstract summary: We present a modular differentiable design that yields performance superior to previous methods by existing, highly optimized graphics pipelines. Our design supports all crucial operations in a modern graphics pipeline: leveraging large numbers of triangles, attribute, filtered texture lookups, as well as user-programmable shading and geometry processing.
• Score: 37.12883917723895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a modular differentiable renderer design that yields performance superior to previous methods by leveraging existing, highly optimized hardware graphics pipelines. Our design supports all crucial operations in a modern graphics pipeline: rasterizing large numbers of triangles, attribute interpolation, filtered texture lookups, as well as user-programmable shading and geometry processing, all in high resolutions. Our modular primitives allow custom, high-performance graphics pipelines to be built directly within automatic differentiation frameworks such as PyTorch or TensorFlow. As a motivating application, we formulate facial performance capture as an inverse rendering problem and show that it can be solved efficiently using our tools. Our results indicate that this simple and straightforward approach achieves excellent geometric correspondence between rendered results and reference imagery.

Related papers

• LinPrim: Linear Primitives for Differentiable Volumetric Rendering [53.780682194322225]
  We introduce two new scene representations based on linear primitives. We present a different octaiableizer that runs efficiently on GPU. We demonstrate comparable performance to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-01-27T18:49:38Z)
• SuperSVG: Superpixel-based Scalable Vector Graphics Synthesis [66.44553285020066]
  SuperSVG is a superpixel-based vectorization model that achieves fast and high-precision image vectorization. We propose a two-stage self-training framework, where a coarse-stage model is employed to reconstruct the main structure and a refinement-stage model is used for enriching the details. Experiments demonstrate the superior performance of our method in terms of reconstruction accuracy and inference time compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-06-14T07:43:23Z)
• Evolutive Rendering Models [91.99498492855187]
  We present textitevolutive rendering models, a methodology where rendering models possess the ability to evolve and adapt dynamically throughout rendering process. In particular, we present a comprehensive learning framework that enables the optimization of three principal rendering elements. A detailed analysis of gradient characteristics is performed to facilitate a stable goal-oriented elements evolution.
  arXiv  Detail & Related papers  (2024-05-27T17:40:00Z)
• FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces [21.946327323788275]
  3D rendering of dynamic face is a challenging problem. We present a novel representation that enables high-quality rendering of an actor's dynamic facial performances.
  arXiv  Detail & Related papers  (2024-04-22T00:44:13Z)
• Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives [70.32817882783608]
  We present an approach that produces a simple, compact, and actionable 3D world representation by means of 3D primitives. Unlike existing primitive decomposition methods that rely on 3D input data, our approach operates directly on images. We show that the resulting textured primitives faithfully reconstruct the input images and accurately model the visible 3D points.
  arXiv  Detail & Related papers  (2023-07-11T17:58:31Z)
• NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Support [45.68296352822415]
  We present a method for generating high-quality watertight manifold meshes from multi-view input images. Our method combines the benefits of both worlds; we take the geometry obtained from neural fields, and further optimize the geometry as well as a compact neural texture representation.
  arXiv  Detail & Related papers  (2023-05-26T17:59:21Z)
• Adaptive Single Image Deblurring [43.02281823557039]
  We propose an efficient pixel adaptive and feature attentive design for handling large blur variations within and across different images. We also propose an effective content-aware global-local filtering module that significantly improves the performance.
  arXiv  Detail & Related papers  (2022-01-01T10:10:19Z)
• Extracting Triangular 3D Models, Materials, and Lighting From Images [59.33666140713829]
  We present an efficient method for joint optimization of materials and lighting from multi-view image observations. We leverage meshes with spatially-varying materials and environment that can be deployed in any traditional graphics engine.
  arXiv  Detail & Related papers  (2021-11-24T13:58:20Z)
• Image Matching across Wide Baselines: From Paper to Practice [80.9424750998559]
  We introduce a comprehensive benchmark for local features and robust estimation algorithms. Our pipeline's modular structure allows easy integration, configuration, and combination of different methods. We show that with proper settings, classical solutions may still outperform the perceived state of the art.
  arXiv  Detail & Related papers  (2020-03-03T15:20:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.