Random-Access Neural Compression of Material Textures
- URL: http://arxiv.org/abs/2305.17105v1
- Date: Fri, 26 May 2023 17:16:22 GMT
- Title: Random-Access Neural Compression of Material Textures
- Authors: Karthik Vaidyanathan, Marco Salvi, Bartlomiej Wronski, Tomas
Akenine-M\"oller, Pontus Ebelin, Aaron Lefohn
- Abstract summary: We propose a novel neural compression technique specifically designed for material textures.
We unlock two more levels of detail, i.e., 16x more texels, using low compression.
Our method allows on-demand, real-time decompression with random access, enabling compression on disk and memory.
- Score: 1.2971248363246106
- License: http://creativecommons.org/licenses/by-sa/4.0/
- Abstract: The continuous advancement of photorealism in rendering is accompanied by a
growth in texture data and, consequently, increasing storage and memory
demands. To address this issue, we propose a novel neural compression technique
specifically designed for material textures. We unlock two more levels of
detail, i.e., 16x more texels, using low bitrate compression, with image
quality that is better than advanced image compression techniques, such as AVIF
and JPEG XL. At the same time, our method allows on-demand, real-time
decompression with random access similar to block texture compression on GPUs,
enabling compression on disk and memory. The key idea behind our approach is
compressing multiple material textures and their mipmap chains together, and
using a small neural network, that is optimized for each material, to
decompress them. Finally, we use a custom training implementation to achieve
practical compression speeds, whose performance surpasses that of general
frameworks, like PyTorch, by an order of magnitude.
Related papers
- Map-Assisted Remote-Sensing Image Compression at Extremely Low Bitrates [47.47031054057152]
Generative models have been explored to compress RS images into extremely low-bitrate streams.
These generative models struggle to reconstruct visually plausible images due to the highly ill-posed nature of extremely low-bitrate image compression.
We propose an image compression framework that utilizes a pre-trained diffusion model with powerful natural image priors to achieve high-realism reconstructions.
arXiv Detail & Related papers (2024-09-03T14:29:54Z) - Neural Graphics Texture Compression Supporting Random Access [34.974631096947284]
We introduce a novel approach to texture set compression that integrates traditional GPU texture representation and NIC techniques.
We propose an asymmetric auto-encoder framework that employs a convolutional encoder to capture detailed information in a bottleneck-latent space.
Experimental results demonstrate that this approach provides much better results than conventional texture compression.
arXiv Detail & Related papers (2024-05-06T19:44:13Z) - Exploring the Limits of Semantic Image Compression at Micro-bits per
Pixel [8.518076792914039]
We use GPT-4V and DALL-E3 from OpenAI to explore the quality-compression frontier for image compression.
We push semantic compression as low as 100 $mu$bpp (up to $10,000times$ smaller than JPEG) by introducing an iterative reflection process.
We further hypothesize this 100 $mu$bpp level represents a soft limit on semantic compression at standard image resolutions.
arXiv Detail & Related papers (2024-02-21T05:14:30Z) - You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
Learned image compression (LIC) methods have experienced significant progress during recent years.
LIC methods fail to explicitly explore the image structure and texture components crucial for image compression.
We present DA-Mask that samples visible patches based on the structure and texture of original images.
We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
arXiv Detail & Related papers (2023-06-27T15:36:22Z) - A Unified Image Preprocessing Framework For Image Compression [5.813935823171752]
We propose a unified image compression preprocessing framework, called Kuchen, to improve the performance of existing codecs.
The framework consists of a hybrid data labeling system along with a learning-based backbone to simulate personalized preprocessing.
Results demonstrate that the modern codecs optimized by our unified preprocessing framework constantly improve the efficiency of the state-of-the-art compression.
arXiv Detail & Related papers (2022-08-15T10:41:00Z) - Optimizing Image Compression via Joint Learning with Denoising [49.83680496296047]
High levels of noise usually exist in today's captured images due to the relatively small sensors equipped in the smartphone cameras.
We propose a novel two-branch, weight-sharing architecture with plug-in feature denoisers to allow a simple and effective realization of the goal with little computational cost.
arXiv Detail & Related papers (2022-07-22T04:23:01Z) - PILC: Practical Image Lossless Compression with an End-to-end GPU
Oriented Neural Framework [88.18310777246735]
We propose an end-to-end image compression framework that achieves 200 MB/s for both compression and decompression with a single NVIDIA Tesla V100 GPU.
Experiments show that our framework compresses better than PNG by a margin of 30% in multiple datasets.
arXiv Detail & Related papers (2022-06-10T03:00:10Z) - How to Exploit the Transferability of Learned Image Compression to
Conventional Codecs [25.622863999901874]
We show how learned image coding can be used as a surrogate to optimize an image for encoding.
Our approach can remodel a conventional image to adjust for the MS-SSIM distortion with over 20% rate improvement without any decoding overhead.
arXiv Detail & Related papers (2020-12-03T12:34:51Z) - Conditional Entropy Coding for Efficient Video Compression [82.35389813794372]
We propose a very simple and efficient video compression framework that only focuses on modeling the conditional entropy between frames.
We first show that a simple architecture modeling the entropy between the image latent codes is as competitive as other neural video compression works and video codecs.
We then propose a novel internal learning extension on top of this architecture that brings an additional 10% savings without trading off decoding speed.
arXiv Detail & Related papers (2020-08-20T20:01:59Z) - Quantization Guided JPEG Artifact Correction [69.04777875711646]
We develop a novel architecture for artifact correction using the JPEG files quantization matrix.
This allows our single model to achieve state-of-the-art performance over models trained for specific quality settings.
arXiv Detail & Related papers (2020-04-17T00:10:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.