SADN: Learned Light Field Image Compression with Spatial-Angular Decorrelation

• URL: http://arxiv.org/abs/2202.10837v1
• Date: Tue, 22 Feb 2022 11:53:52 GMT
• Title: SADN: Learned Light Field Image Compression with Spatial-Angular Decorrelation
• Authors: Kedeng Tong, Xin Jin, Chen Wang, Fan Jiang
• Abstract summary: We propose a novel end-to-end spatial-angular-decorrelated network (SADN) for high-efficiency light field image compression. The proposed method provides 2.137 times and 2.849 times higher compression efficiency relative to H.266/VVC and H.265/HEVC inter coding. It also outperforms the end-to-end image compression networks by an average of 79.6% saving with much higher subjective quality and light field consistency.
• Score: 15.262518233068622
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Light field image becomes one of the most promising media types for immersive video applications. In this paper, we propose a novel end-to-end spatial-angular-decorrelated network (SADN) for high-efficiency light field image compression. Different from the existing methods that exploit either spatial or angular consistency in the light field image, SADN decouples the angular and spatial information by dilation convolution and stride convolution in spatial-angular interaction, and performs feature fusion to compress spatial and angular information jointly. To train a stable and robust algorithm, a large-scale dataset consisting of 7549 light field images is proposed and built. The proposed method provides 2.137 times and 2.849 times higher compression efficiency relative to H.266/VVC and H.265/HEVC inter coding, respectively. It also outperforms the end-to-end image compression networks by an average of 79.6% bitrate saving with much higher subjective quality and light field consistency.

Related papers

• MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model [78.4051835615796]
  This paper proposes a method called Multimodal Image Semantic Compression. It consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information. It can achieve optimal consistency and perception results while saving perceptual 50%, which has strong potential applications in the next generation of storage and communication.
  arXiv  Detail & Related papers  (2024-02-26T17:11:11Z)
• OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation [48.828453331724965]
  We propose an Omni-Aperture Fusion model (OAFuser) to extract angular information from sub-aperture images to generate semantically consistent results. The proposed OAFuser achieves state-of-the-art performance on four UrbanLF datasets in terms of all evaluation metrics.
  arXiv  Detail & Related papers  (2023-07-28T14:43:27Z)
• Learned Focused Plenoptic Image Compression with Microimage Preprocessing and Global Attention [17.05466366805901]
  Focused plenoptic cameras can record spatial and angular information of the light field (LF) simultaneously. The existing plenoptic image compression methods present ineffectiveness to the captured images due to the complex micro-textures generated by the microlens relay imaging and long-distance correlations among the microimages. A lossy end-to-end learning architecture is proposed to compress the focused plenoptic images efficiently.
  arXiv  Detail & Related papers  (2023-04-30T14:24:56Z)
• Learning-based Spatial and Angular Information Separation for Light Field Compression [29.827366575505557]
  We propose a novel neural network that can separate angular and spatial information of a light field. The network represents spatial information using spatial kernels shared among all Sub-Aperture Images (SAIs), and angular information using sets of angular kernels for each SAI.
  arXiv  Detail & Related papers  (2023-04-13T08:02:38Z)
• Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution [36.69391399634076]
  Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR) We propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR. Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line.
  arXiv  Detail & Related papers  (2023-02-16T03:40:40Z)
• Learned Distributed Image Compression with Multi-Scale Patch Matching in Feature Domai [62.88240343479615]
  We propose Multi-Scale Feature Domain Patch Matching (MSFDPM) to fully utilize side information at the decoder of the distributed image compression model. MSFDPM consists of a side information feature extractor, a multi-scale feature domain patch matching module, and a multi-scale feature fusion network. Our patch matching in a multi-scale feature domain further improves compression rate by about 20% compared with the patch matching method at image domain.
  arXiv  Detail & Related papers  (2022-09-06T14:06:46Z)
• Efficient Light Field Reconstruction via Spatio-Angular Dense Network [14.568586050271357]
  We propose an end-to-end Spatio-Angular Dense Network (SADenseNet) for light field reconstruction. We show that the proposed SADenseNet's state-of-the-art performance at significantly reduced costs in memory and computation. Results show that the reconstructed light field images are sharp with correct details and can serve as pre-processing to improve the accuracy of measurement related applications.
  arXiv  Detail & Related papers  (2021-08-08T13:50:51Z)
• Learning-Based Practical Light Field Image Compression Using A Disparity-Aware Model [1.5229257192293197]
  We propose a new learning-based, disparity-aided model for compression of 4D light field images. The model is end-to-end trainable, eliminating the need for hand-tuning separate modules. Comparisons with the state of the art show encouraging performance in terms of PSNR and MS-SSIM metrics.
  arXiv  Detail & Related papers  (2021-06-22T06:30:25Z)
• Low Light Image Enhancement via Global and Local Context Modeling [164.85287246243956]
  We introduce a context-aware deep network for low-light image enhancement. First, it features a global context module that models spatial correlations to find complementary cues over full spatial domain. Second, it introduces a dense residual block that captures local context with a relatively large receptive field.
  arXiv  Detail & Related papers  (2021-01-04T09:40:54Z)
• Spatial-Angular Attention Network for Light Field Reconstruction [64.27343801968226]
  We propose a spatial-angular attention network to perceive correspondences in the light field non-locally. Motivated by the non-local attention mechanism, a spatial-angular attention module is introduced to compute the responses from all the positions in the epipolar plane for each pixel in the light field. We then propose a multi-scale reconstruction structure to efficiently implement the non-local attention in the low spatial scale.
  arXiv  Detail & Related papers  (2020-07-05T06:55:29Z)
• Learning Light Field Angular Super-Resolution via a Geometry-Aware Network [101.59693839475783]
  We propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48$times$.
  arXiv  Detail & Related papers  (2020-02-26T02:36:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.