Neural Network Assisted Depth Map Packing for Compression Using Standard Hardware Video Codecs

• URL: http://arxiv.org/abs/2206.15183v1
• Date: Thu, 30 Jun 2022 10:46:05 GMT
• Title: Neural Network Assisted Depth Map Packing for Compression Using Standard Hardware Video Codecs
• Authors: Matti Siekkinen and Teemu K\"am\"ar\"ainen
• Abstract summary: We propose a variable precision packing scheme assisted by a neural network model that predicts the optimal precision for each depth map given a constraint. We demonstrate that the model yields near optimal predictions and that it can be integrated into a game engine with very low overhead.
• Score: 0.38073142980732994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Depth maps are needed by various graphics rendering and processing operations. Depth map streaming is often necessary when such operations are performed in a distributed system and it requires in most cases fast performing compression, which is why video codecs are often used. Hardware implementations of standard video codecs enable relatively high resolution and framerate combinations, even on resource constrained devices, but unfortunately those implementations do not currently support RGB+depth extensions. However, they can be used for depth compression by first packing the depth maps into RGB or YUV frames. We investigate depth map compression using a combination of depth map packing followed by encoding with a standard video codec. We show that the precision at which depth maps are packed has a large and nontrivial impact on the resulting error caused by the combination of the packing scheme and lossy compression when bitrate is constrained. Consequently, we propose a variable precision packing scheme assisted by a neural network model that predicts the optimal precision for each depth map given a bitrate constraint. We demonstrate that the model yields near optimal predictions and that it can be integrated into a game engine with very low overhead using modern hardware.

Related papers

• Embedding Compression Distortion in Video Coding for Machines [67.97469042910855]
  Currently, video transmission serves not only the Human Visual System (HVS) for viewing but also machine perception for analysis. We propose a Compression Distortion Embedding (CDRE) framework, which extracts machine-perception-related distortion representation and embeds it into downstream models. Our framework can effectively boost the rate-task performance of existing codecs with minimal overhead in terms of execution time, and number of parameters.
  arXiv  Detail & Related papers  (2025-03-27T13:01:53Z)
• Unified Dense Prediction of Video Diffusion [91.16237431830417]
  We present a unified network for simultaneously generating videos and their corresponding entity segmentation and depth maps from text prompts. We utilize colormap to represent entity masks and depth maps, tightly integrating dense prediction with RGB video generation.
  arXiv  Detail & Related papers  (2025-03-12T12:41:02Z)
• High-Efficiency Neural Video Compression via Hierarchical Predictive Learning [27.41398149573729]
  Enhanced Deep Hierarchical Video Compression-DHVC 2.0- introduces superior compression performance and impressive complexity efficiency. Uses hierarchical predictive coding to transform each video frame into multiscale representations. Supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss.
  arXiv  Detail & Related papers  (2024-10-03T15:40:58Z)
• Map-Assisted Remote-Sensing Image Compression at Extremely Low Bitrates [47.47031054057152]
  Generative models have been explored to compress RS images into extremely low-bitrate streams. These generative models struggle to reconstruct visually plausible images due to the highly ill-posed nature of extremely low-bitrate image compression. We propose an image compression framework that utilizes a pre-trained diffusion model with powerful natural image priors to achieve high-realism reconstructions.
  arXiv  Detail & Related papers  (2024-09-03T14:29:54Z)
• A Perspective on Deep Vision Performance with Standard Image and Video Codecs [41.73262031925552]
  Resource-constrained hardware, such as edge devices or cell phones, often rely on cloud servers to provide the required computational resources for inference in deep vision models. This paper aims to examine the implications of employing standardized codecs within deep vision pipelines. We find that using JPEG and H.264 coding significantly deteriorates the accuracy across a broad range of vision tasks and models.
  arXiv  Detail & Related papers  (2024-04-18T16:58:05Z)
• MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model [78.4051835615796]
  This paper proposes a method called Multimodal Image Semantic Compression. It consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information. It can achieve optimal consistency and perception results while saving perceptual 50%, which has strong potential applications in the next generation of storage and communication.
  arXiv  Detail & Related papers  (2024-02-26T17:11:11Z)
• You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
  Learned image compression (LIC) methods have experienced significant progress during recent years. LIC methods fail to explicitly explore the image structure and texture components crucial for image compression. We present DA-Mask that samples visible patches based on the structure and texture of original images. We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
  arXiv  Detail & Related papers  (2023-06-27T15:36:22Z)
• Block Modulating Video Compression: An Ultra Low Complexity Image Compression Encoder for Resource Limited Platforms [35.76050232152349]
  An ultra low-cost image Modulating Video Compression (BMVC) is proposed to be implemented on mobile platforms with low consumption of power and computation resources. Two types of BMVC decoders, implemented by deep neural networks, are presented.
  arXiv  Detail & Related papers  (2022-05-07T16:20:09Z)
• End-to-end lossless compression of high precision depth maps guided by pseudo-residual [6.213322670014608]
  It is urgent to explore a new compression method with better compression ratio for high precision depth maps. We propose an end-to-end learning-based lossless compression method for high precision depth maps.
  arXiv  Detail & Related papers  (2022-01-10T07:19:02Z)
• A New Image Codec Paradigm for Human and Machine Uses [53.48873918537017]
  A new scalable image paradigm for both human and machine uses is proposed in this work. The high-level instance segmentation map and the low-level signal features are extracted with neural networks. An image is designed and trained to achieve the general-quality image reconstruction with the 16-bit gray-scale profile and signal features.
  arXiv  Detail & Related papers  (2021-12-19T06:17:38Z)
• Deep Contextual Video Compression [20.301569390401102]
  We propose a deep contextual video compression framework to enable a paradigm shift from predictive coding to conditional coding. Our method can significantly outperform the previous state-of-theart (SOTA) deep video compression methods.
  arXiv  Detail & Related papers  (2021-09-30T12:14:24Z)
• Learning to Localize Through Compressed Binary Maps [83.03367511221437]
  We learn to compress the map representation such that it is optimal for the localization task. Our experiments show that it is possible to learn a task-specific compression which reduces storage requirements by two orders of magnitude over general-purpose codecs.
  arXiv  Detail & Related papers  (2020-12-20T14:47:15Z)
• Content Adaptive and Error Propagation Aware Deep Video Compression [110.31693187153084]
  We propose a content adaptive and error propagation aware video compression system. Our method employs a joint training strategy by considering the compression performance of multiple consecutive frames instead of a single frame. Instead of using the hand-crafted coding modes in the traditional compression systems, we design an online encoder updating scheme in our system.
  arXiv  Detail & Related papers  (2020-03-25T09:04:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.