NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement

• URL: http://arxiv.org/abs/2306.11920v3
• Date: Sun, 24 Dec 2023 13:12:55 GMT
• Title: NILUT: Conditional Neural Implicit 3D Lookup Tables for Image Enhancement
• Authors: Marcos V. Conde, Javier Vazquez-Corral, Michael S. Brown, Radu Timofte
• Abstract summary: 3D lookup tables (3D LUTs) are a key component for image enhancement. Current approaches for learning and applying 3D LUTs are notably fast, yet not so memory-efficient. We propose a Neural Implicit LUT (NILUT), an implicitly defined continuous 3D color transformation parameterized by a neural network.
• Score: 82.75363196702381
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: 3D lookup tables (3D LUTs) are a key component for image enhancement. Modern image signal processors (ISPs) have dedicated support for these as part of the camera rendering pipeline. Cameras typically provide multiple options for picture styles, where each style is usually obtained by applying a unique handcrafted 3D LUT. Current approaches for learning and applying 3D LUTs are notably fast, yet not so memory-efficient, as storing multiple 3D LUTs is required. For this reason and other implementation limitations, their use on mobile devices is less popular. In this work, we propose a Neural Implicit LUT (NILUT), an implicitly defined continuous 3D color transformation parameterized by a neural network. We show that NILUTs are capable of accurately emulating real 3D LUTs. Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly. Our novel approach is memory-efficient, controllable and can complement previous methods, including learned ISPs. Code, models and dataset available at: https://github.com/mv-lab/nilut

Related papers

• Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image [80.48452783328995]
  Flash3D is a method for scene reconstruction and novel view synthesis from a single image. For generalisability, we start from a "foundation" model for monocular depth estimation. For efficiency, we base this extension on feed-forward Gaussian Splatting.
  arXiv  Detail & Related papers  (2024-06-06T17:59:56Z)
• An intuitive multi-frequency feature representation for SO(3)-equivariant networks [9.092163300680832]
  We introduce an equivariant feature representation for mapping a 3D point to a high-dimensional feature space. Our representation can be used as an input to VNs, and the results demonstrate that with our feature representation, VN captures more details.
  arXiv  Detail & Related papers  (2024-03-15T11:36:50Z)
• Free3D: Consistent Novel View Synthesis without 3D Representation [63.931920010054064]
  Free3D is a simple accurate method for monocular open-set novel view synthesis (NVS) Compared to other works that took a similar approach, we obtain significant improvements without resorting to an explicit 3D representation.
  arXiv  Detail & Related papers  (2023-12-07T18:59:18Z)
• WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space [77.92350895927922]
  We propose WildFusion, a new approach to 3D-aware image synthesis based on latent diffusion models (LDMs) Our 3D-aware LDM is trained without any direct supervision from multiview images or 3D geometry. This opens up promising research avenues for scalable 3D-aware image synthesis and 3D content creation from in-the-wild image data.
  arXiv  Detail & Related papers  (2023-11-22T18:25:51Z)
• Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations [92.88108411154255]
  We present a method that improves dense 2D image feature extractors when the latter are applied to the analysis of multiple images reconstructible as a 3D scene. We show that our method not only enables semantic understanding in the context of scene-specific neural fields without the use of manual labels, but also consistently improves over the self-supervised 2D baselines.
  arXiv  Detail & Related papers  (2022-09-07T23:24:09Z)
• Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions [70.04394678730968]
  We present an approach for the reconstruction of 3D human heads from one or few views. The underlying neural architecture is to learn the objects and to generalize the model. Our model can fit novel heads on just a hundred videos or one-shot 3D scans.
  arXiv  Detail & Related papers  (2022-09-07T21:09:24Z)
• Learning Image-adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-time [33.93249921871407]
  In this paper, we learn image-adaptive 3-dimensional lookup tables (3D LUTs) to achieve fast and robust photo enhancement. We learn 3D LUTs from annotated data using pairwise or unpaired learning. We learn multiple basis 3D LUTs and a small convolutional neural network (CNN) simultaneously in an end-to-end manner.
  arXiv  Detail & Related papers  (2020-09-30T06:34:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.