Raw Image Reconstruction with Learned Compact Metadata

• URL: http://arxiv.org/abs/2302.12995v2
• Date: Tue, 28 Feb 2023 03:48:03 GMT
• Title: Raw Image Reconstruction with Learned Compact Metadata
• Authors: Yufei Wang, Yi Yu, Wenhan Yang, Lanqing Guo, Lap-Pui Chau, Alex Kot, Bihan Wen
• Abstract summary: We propose a novel framework to learn a compact representation in the latent space serving as the metadata in an end-to-end manner. We show how the proposed raw image compression scheme can adaptively allocate more bits to image regions that are important from a global perspective.
• Score: 61.62454853089346
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While raw images exhibit advantages over sRGB images (e.g., linearity and fine-grained quantization level), they are not widely used by common users due to the large storage requirements. Very recent works propose to compress raw images by designing the sampling masks in the raw image pixel space, leading to suboptimal image representations and redundant metadata. In this paper, we propose a novel framework to learn a compact representation in the latent space serving as the metadata in an end-to-end manner. Furthermore, we propose a novel sRGB-guided context model with improved entropy estimation strategies, which leads to better reconstruction quality, smaller size of metadata, and faster speed. We illustrate how the proposed raw image compression scheme can adaptively allocate more bits to image regions that are important from a global perspective. The experimental results show that the proposed method can achieve superior raw image reconstruction results using a smaller size of the metadata on both uncompressed sRGB images and JPEG images.

Related papers

• A Learnable Color Correction Matrix for RAW Reconstruction [19.394856071610604]
  We introduce a learnable color correction matrix (CCM) to approximate the complex inverse image signal processor (ISP) Experimental results demonstrate that simulated RAW (simRAW) images generated by our method provide performance improvements equivalent to those produced by more complex inverse ISP methods.
  arXiv  Detail & Related papers  (2024-09-04T07:46:42Z)
• You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
  Learned image compression (LIC) methods have experienced significant progress during recent years. LIC methods fail to explicitly explore the image structure and texture components crucial for image compression. We present DA-Mask that samples visible patches based on the structure and texture of original images. We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
  arXiv  Detail & Related papers  (2023-06-27T15:36:22Z)
• Beyond Learned Metadata-based Raw Image Reconstruction [86.1667769209103]
  Raw images have distinct advantages over sRGB images, e.g., linearity and fine-grained quantization levels. They are not widely adopted by general users due to their substantial storage requirements. We propose a novel framework that learns a compact representation in the latent space, serving as metadata.
  arXiv  Detail & Related papers  (2023-06-21T06:59:07Z)
• {\mu}Split: efficient image decomposition for microscopy data [50.794670705085835]
  muSplit is a dedicated approach for trained image decomposition in the context of fluorescence microscopy images. We introduce lateral contextualization (LC), a novel meta-architecture that enables the memory efficient incorporation of large image-context. We apply muSplit to five decomposition tasks, one on a synthetic dataset, four others derived from real microscopy data.
  arXiv  Detail & Related papers  (2022-11-23T11:26:24Z)
• Learning sRGB-to-Raw-RGB De-rendering with Content-Aware Metadata [46.28281823015191]
  We show how to improve the de-rendering results by jointly learning sampling and reconstruction. Our experiments show that our learned sampling can adapt to the image content to produce better raw reconstructions than existing methods.
  arXiv  Detail & Related papers  (2022-06-03T20:43:17Z)
• Towards Low Light Enhancement with RAW Images [101.35754364753409]
  We make the first benchmark effort to elaborate on the superiority of using RAW images in the low light enhancement. We develop a new evaluation framework, Factorized Enhancement Model (FEM), which decomposes the properties of RAW images into measurable factors. A RAW-guiding Exposure Enhancement Network (REENet) is developed, which makes trade-offs between the advantages and inaccessibility of RAW images in real applications.
  arXiv  Detail & Related papers  (2021-12-28T07:27:51Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)
• Invertible Image Signal Processing [42.109752151834456]
  Invertible Image Signal Processing (InvISP) pipeline enables rendering visually appealing sRGB images. We can reconstruct realistic RAW data instead of synthesizing RAW data from sRGB images without any memory overhead.
  arXiv  Detail & Related papers  (2021-03-28T06:30:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.