A Hierarchical Coding Scheme for Glasses-free 3D Displays Based on Scalable Hybrid Layered Representation of Real-World Light Fields

• URL: http://arxiv.org/abs/2104.09378v1
• Date: Mon, 19 Apr 2021 15:09:21 GMT
• Title: A Hierarchical Coding Scheme for Glasses-free 3D Displays Based on Scalable Hybrid Layered Representation of Real-World Light Fields
• Authors: Joshitha R and Mansi Sharma
• Abstract summary: Scheme learns stacked multiplicative layers from subsets of light field views determined from different scanning orders. The spatial correlation in layer patterns is exploited with varying low ranks in factorization derived from singular value decomposition on a Krylov subspace. encoding with HEVC efficiently removes intra-view and inter-view correlation in low-rank approximated layers.
• Score: 0.6091702876917279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper presents a novel hierarchical coding scheme for light fields based on transmittance patterns of low-rank multiplicative layers and Fourier disparity layers. The proposed scheme learns stacked multiplicative layers from subsets of light field views determined from different scanning orders. The multiplicative layers are optimized using a fast data-driven convolutional neural network (CNN). The spatial correlation in layer patterns is exploited with varying low ranks in factorization derived from singular value decomposition on a Krylov subspace. Further, encoding with HEVC efficiently removes intra-view and inter-view correlation in low-rank approximated layers. The initial subset of approximated decoded views from multiplicative representation is used to construct Fourier disparity layer (FDL) representation. The FDL model synthesizes second subset of views which is identified by a pre-defined hierarchical prediction order. The correlations between the prediction residue of synthesized views is further eliminated by encoding the residual signal. The set of views obtained from decoding the residual is employed in order to refine the FDL model and predict the next subset of views with improved accuracy. This hierarchical procedure is repeated until all light field views are encoded. The critical advantage of proposed hybrid layered representation and coding scheme is that it utilizes not just spatial and temporal redundancies, but efficiently exploits the strong intrinsic similarities among neighboring sub-aperture images in both horizontal and vertical directions as specified by different predication orders. Besides, the scheme is flexible to realize a range of multiple bitrates at the decoder within a single integrated system. The compression performance analyzed with real light field shows substantial bitrate savings, maintaining good reconstruction quality.

Related papers

• Multiscale Graph Neural Network Autoencoders for Interpretable Scientific Machine Learning [0.0]
  The goal of this work is to address two limitations in autoencoder-based models: latent space interpretability and compatibility with unstructured meshes. This is accomplished here with the development of a novel graph neural network (GNN) autoencoding architecture with demonstrations on complex fluid flow applications.
  arXiv  Detail & Related papers  (2023-02-13T08:47:11Z)
• WLD-Reg: A Data-dependent Within-layer Diversity Regularizer [98.78384185493624]
  Neural networks are composed of multiple layers arranged in a hierarchical structure jointly trained with a gradient-based optimization. We propose to complement this traditional 'between-layer' feedback with additional 'within-layer' feedback to encourage the diversity of the activations within the same layer. We present an extensive empirical study confirming that the proposed approach enhances the performance of several state-of-the-art neural network models in multiple tasks.
  arXiv  Detail & Related papers  (2023-01-03T20:57:22Z)
• Semantic Image Synthesis via Diffusion Models [159.4285444680301]
  Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks. Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
  arXiv  Detail & Related papers  (2022-06-30T18:31:51Z)
• PINs: Progressive Implicit Networks for Multi-Scale Neural Representations [68.73195473089324]
  We propose a progressive positional encoding, exposing a hierarchical structure to incremental sets of frequency encodings. Our model accurately reconstructs scenes with wide frequency bands and learns a scene representation at progressive level of detail. Experiments on several 2D and 3D datasets show improvements in reconstruction accuracy, representational capacity and training speed compared to baselines.
  arXiv  Detail & Related papers  (2022-02-09T20:33:37Z)
• A Novel Hierarchical Light Field Coding Scheme Based on Hybrid Stacked Multiplicative Layers and Fourier Disparity Layers for Glasses-Free 3D Displays [0.6091702876917279]
  We present a novel hierarchical coding scheme for light fields based on transmittance patterns of low-rank multiplicative layers and Fourier disparity layers. The proposed scheme identifies multiplicative layers of light field view subsets optimized using a convolutional neural network for different scanning orders.
  arXiv  Detail & Related papers  (2021-08-27T17:09:29Z)
• Modulated Periodic Activations for Generalizable Local Functional Representations [113.64179351957888]
  We present a new representation that generalizes to multiple instances and achieves state-of-the-art fidelity. Our approach produces general functional representations of images, videos and shapes, and achieves higher reconstruction quality than prior works that are optimized for a single signal.
  arXiv  Detail & Related papers  (2021-04-08T17:59:04Z)
• Layer Decomposition Learning Based on Gaussian Convolution Model and Residual Deblurring for Inverse Halftoning [7.462336024223669]
  Layer decomposition to separate an input image into base and detail layers has been steadily used for image restoration. In inverse halftoning, homogenous dot patterns hinder a small output range from the residual layers. A new layer decomposition network based on the Gaussian convolution model (GCM) and structure-aware deblurring strategy is presented.
  arXiv  Detail & Related papers  (2020-12-27T09:15:00Z)
• Dual-constrained Deep Semi-Supervised Coupled Factorization Network with Enriched Prior [80.5637175255349]
  We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net. To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network. Our network can obtain state-of-the-art performance for representation learning and clustering.
  arXiv  Detail & Related papers  (2020-09-08T13:10:21Z)
• Rethinking and Improving Natural Language Generation with Layer-Wise Multi-View Decoding [59.48857453699463]
  In sequence-to-sequence learning, the decoder relies on the attention mechanism to efficiently extract information from the encoder. Recent work has proposed to use representations from different encoder layers for diversified levels of information. We propose layer-wise multi-view decoding, where for each decoder layer, together with the representations from the last encoder layer, which serve as a global view, those from other encoder layers are supplemented for a stereoscopic view of the source sequences.
  arXiv  Detail & Related papers  (2020-05-16T20:00:39Z)
• Generalized Octave Convolutions for Learned Multi-Frequency Image Compression [20.504561050200365]
  We propose the first learned multi-frequency image compression and entropy coding approach. It is based on the recently developed octave convolutions to factorize the latents into high and low frequency (resolution) components. We show that the proposed generalized octave convolution can improve the performance of other auto-encoder-based computer vision tasks.
  arXiv  Detail & Related papers  (2020-02-24T01:35:29Z)
• Multi-Level Representation Learning for Deep Subspace Clustering [10.506584969668792]
  This paper proposes a novel deep subspace clustering approach which uses convolutional autoencoders to transform input images into new representations lying on a union of linear subspaces. Experiments on four real-world datasets demonstrate that our approach exhibits superior performance compared to the state-of-the-art methods on most of the subspace clustering problems.
  arXiv  Detail & Related papers  (2020-01-19T23:29:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.