An Integrated Representation & Compression Scheme Based on Convolutional Autoencoders with 4D DCT Perceptual Encoding for High Dynamic Range Light Fields

• URL: http://arxiv.org/abs/2206.10131v1
• Date: Tue, 21 Jun 2022 06:25:06 GMT
• Title: An Integrated Representation & Compression Scheme Based on Convolutional Autoencoders with 4D DCT Perceptual Encoding for High Dynamic Range Light Fields
• Authors: Sally Khaidem and Mansi Sharma
• Abstract summary: Light field size is a major drawback while utilising 3D displays and streaming purposes. In this paper, we propose a novel compression algorithm for a high dynamic range light field. The algorithm exploits the inter and intra view correlations of the HDR light field by interpreting it to be a four-dimension volume.
• Score: 0.30458514384586394
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The emerging and existing light field displays are highly capable of realistic presentation of 3D scenes on auto-stereoscopic glasses-free platforms. The light field size is a major drawback while utilising 3D displays and streaming purposes. When a light field is of high dynamic range, the size increases drastically. In this paper, we propose a novel compression algorithm for a high dynamic range light field which yields a perceptually lossless compression. The algorithm exploits the inter and intra view correlations of the HDR light field by interpreting it to be a four-dimension volume. The HDR light field compression is based on a novel 4DDCT-UCS (4D-DCT Uniform Colour Space) algorithm. Additional encoding of 4DDCT-UCS acquired images by HEVC eliminates intra-frame, inter-frame and intrinsic redundancies in HDR light field data. Comparison with state-of-the-art coders like JPEG-XL and HDR video coding algorithm exhibits superior compression performance of the proposed scheme for real-world light fields.

Related papers

• HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting [76.5908492298286]
  Existing HDR NVS methods are mainly based on NeRF. Existing HDR NVS methods are mainly based on NeRF. They suffer from long training time and slow inference speed. We propose a new framework, High Dynamic Range Gaussian Splatting (-GS), which can efficiently render novel HDR views.
  arXiv  Detail & Related papers  (2024-05-24T00:46:58Z)
• 4K4D: Real-Time 4D View Synthesis at 4K Resolution [86.6582179227016]
  This paper targets high-fidelity and real-time view of dynamic 3D scenes at 4K resolution. We propose a 4D point cloud representation that supports hardwareization and enables unprecedented rendering speed. Our representation can be rendered at over 400 FPS on the DNA-Rendering dataset at 1080p resolution and 80 FPS on the ENeRF-Outdoor dataset at 4K resolution using an 4090 GPU.
  arXiv  Detail & Related papers  (2023-10-17T17:57:38Z)
• Learning Kernel-Modulated Neural Representation for Efficient Light Field Compression [41.24757573290883]
  We design a compact neural network representation for the light field compression task. It is composed of two types of complementary kernels: descriptive kernels (descriptors) that store scene description information learned during training, and modulatory kernels (modulators) that control the rendering of different SAIs from the queried perspectives.
  arXiv  Detail & Related papers  (2023-07-12T12:58:03Z)
• Spatiotemporally Consistent HDR Indoor Lighting Estimation [66.26786775252592]
  We propose a physically-motivated deep learning framework to solve the indoor lighting estimation problem. Given a single LDR image with a depth map, our method predicts spatially consistent lighting at any given image position. Our framework achieves photorealistic lighting prediction with higher quality compared to state-of-the-art single-image or video-based methods.
  arXiv  Detail & Related papers  (2023-05-07T20:36:29Z)
• Real-Time Radiance Fields for Single-Image Portrait View Synthesis [85.32826349697972]
  We present a one-shot method to infer and render a 3D representation from a single unposed image in real-time. Given a single RGB input, our image encoder directly predicts a canonical triplane representation of a neural radiance field for 3D-aware novel view synthesis via volume rendering. Our method is fast (24 fps) on consumer hardware, and produces higher quality results than strong GAN-inversion baselines that require test-time optimization.
  arXiv  Detail & Related papers  (2023-05-03T17:56:01Z)
• NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields [99.57774680640581]
  We present an efficient framework capable of fast reconstruction, compact modeling, and streamable rendering. We propose to decompose the 4D space according to temporal characteristics. Points in the 4D space are associated with probabilities belonging to three categories: static, deforming, and new areas.
  arXiv  Detail & Related papers  (2022-10-28T07:11:05Z)
• A Novel Light Field Coding Scheme Based on Deep Belief Network & Weighted Binary Images for Additive Layered Displays [0.30458514384586394]
  Stacking light attenuating layers is one approach to implement a light field display with a broader depth of field, wide viewing angles and high resolution. This paper proposes a novel framework for light field representation and coding that utilizes Deep Belief Network (DBN) and weighted binary images.
  arXiv  Detail & Related papers  (2022-10-04T08:18:06Z)
• RIDDLE: Lidar Data Compression with Range Image Deep Delta Encoding [21.70770383279559]
  Lidars are depth measuring sensors widely used in autonomous driving and augmented reality. Large volume of data produced by lidars can lead to high costs in data storage and transmission. We propose a novel data-driven range image compression algorithm, named RIDDLE (Range Image Deep DeLta.
  arXiv  Detail & Related papers  (2022-06-02T21:53:43Z)
• Learning-Based Practical Light Field Image Compression Using A Disparity-Aware Model [1.5229257192293197]
  We propose a new learning-based, disparity-aided model for compression of 4D light field images. The model is end-to-end trainable, eliminating the need for hand-tuning separate modules. Comparisons with the state of the art show encouraging performance in terms of PSNR and MS-SSIM metrics.
  arXiv  Detail & Related papers  (2021-06-22T06:30:25Z)
• NeLF: Practical Novel View Synthesis with Neural Light Field [93.41020940730915]
  We present a practical and robust deep learning solution for the novel view synthesis of complex scenes. In our approach, a continuous scene is represented as a light field, i.e., a set of rays, each of which has a corresponding color. Our method achieves state-of-the-art novel view synthesis results while maintaining an interactive frame rate.
  arXiv  Detail & Related papers  (2021-05-15T01:20:30Z)
• A Novel Unified Model for Multi-exposure Stereo Coding Based on Low Rank Tucker-ALS and 3D-HEVC [0.6091702876917279]
  We propose an efficient scheme for coding multi-exposure stereo images based on a tensor low-rank approximation scheme. The multi-exposure fusion can be realized to generate HDR stereo output at the decoder for increased realism and binocular 3D depth cues. The encoding with 3D-HEVC enhance the proposed scheme efficiency by exploiting intra-frame, inter-view and the inter-component redundancies in lowrank approximated representation.
  arXiv  Detail & Related papers  (2021-04-10T10:10:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.