Rethinking RGB Color Representation for Image Restoration Models

• URL: http://arxiv.org/abs/2402.03399v1
• Date: Mon, 5 Feb 2024 06:38:39 GMT
• Title: Rethinking RGB Color Representation for Image Restoration Models
• Authors: Jaerin Lee, JoonKyu Park, Sungyong Baik and Kyoung Mu Lee
• Abstract summary: We augment the representation to hold structural information of local neighborhoods at each pixel. Substituting the underlying representation space for the per-pixel losses facilitates the training of image restoration models. Our space consistently improves overall metrics by reconstructing both color and local structures.
• Score: 55.81013540537963
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image restoration models are typically trained with a pixel-wise distance loss defined over the RGB color representation space, which is well known to be a source of blurry and unrealistic textures in the restored images. The reason, we believe, is that the three-channel RGB space is insufficient for supervising the restoration models. To this end, we augment the representation to hold structural information of local neighborhoods at each pixel while keeping the color information and pixel-grainedness unharmed. The result is a new representation space, dubbed augmented RGB ($a$RGB) space. Substituting the underlying representation space for the per-pixel losses facilitates the training of image restoration models, thereby improving the performance without affecting the evaluation phase. Notably, when combined with auxiliary objectives such as adversarial or perceptual losses, our $a$RGB space consistently improves overall metrics by reconstructing both color and local structures, overcoming the conventional perception-distortion trade-off.

Related papers

• Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction [51.87279764576998]
  We propose EvRGBHand -- the first approach for 3D hand mesh reconstruction with an event camera and an RGB camera compensating for each other. EvRGBHand can tackle overexposure and motion blur issues in RGB-based HMR and foreground scarcity and background overflow issues in event-based HMR.
  arXiv  Detail & Related papers  (2024-03-12T06:04:50Z)
• Spherical Space Feature Decomposition for Guided Depth Map Super-Resolution [123.04455334124188]
  Guided depth map super-resolution (GDSR) aims to upsample low-resolution (LR) depth maps with additional information involved in high-resolution (HR) RGB images from the same scene. In this paper, we propose the Spherical Space feature Decomposition Network (SSDNet) to solve the above issues. Our method can achieve state-of-the-art results on four test datasets, as well as successfully generalize to real-world scenes.
  arXiv  Detail & Related papers  (2023-03-15T21:22:21Z)
• Total Scale: Face-to-Body Detail Reconstruction from Sparse RGBD Sensors [52.38220261632204]
  Flat facial surfaces frequently occur in the PIFu-based reconstruction results. We propose a two-scale PIFu representation to enhance the quality of the reconstructed facial details. Experiments demonstrate the effectiveness of our approach in vivid facial details and deforming body shapes.
  arXiv  Detail & Related papers  (2021-12-03T18:46:49Z)
• RGB-D Image Inpainting Using Generative Adversarial Network with a Late Fusion Approach [14.06830052027649]
  Diminished reality is a technology that aims to remove objects from video images and fills in the missing region with plausible pixels. We propose an RGB-D image inpainting method using generative adversarial network.
  arXiv  Detail & Related papers  (2021-10-14T14:44:01Z)
• Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision [76.41657124981549]
  This paper presents a joint learning model for image alignment and RAW-to-sRGB mapping. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets.
  arXiv  Detail & Related papers  (2021-08-18T12:41:36Z)
• Semantic-embedded Unsupervised Spectral Reconstruction from Single RGB Images in the Wild [48.44194221801609]
  We propose a new lightweight and end-to-end learning-based framework to tackle this challenge. We progressively spread the differences between input RGB images and re-projected RGB images from recovered HS images via effective camera spectral response function estimation. Our method significantly outperforms state-of-the-art unsupervised methods and even exceeds the latest supervised method under some settings.
  arXiv  Detail & Related papers  (2021-08-15T05:19:44Z)
• NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image [34.79657678041356]
  We propose a fast adversarial learning-based method to reconstruct the complete and detailed 3D human from a single RGB-D image. Given a consumer RGB-D sensor, NormalGAN can generate the complete and detailed 3D human reconstruction results in 20 fps.
  arXiv  Detail & Related papers  (2020-07-30T09:35:46Z)
• SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans [34.397726189729994]
  SPSG is a novel approach to generate high-quality, colored 3D models of scenes from RGB-D scan observations. Our self-supervised approach learns to jointly inpaint geometry and color by correlating an incomplete RGB-D scan with a more complete version of that scan.
  arXiv  Detail & Related papers  (2020-06-25T18:58:23Z)
• Hierarchical Regression Network for Spectral Reconstruction from RGB Images [21.551899202524904]
  We propose a 4-level Hierarchical Regression Network (HRNet) with PixelShuffle layer as inter-level interaction. We evaluate proposed HRNet with other architectures and techniques by participating in NTIRE 2020 Challenge on Spectral Reconstruction from RGB Images.
  arXiv  Detail & Related papers  (2020-05-10T16:06:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.