Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer

• URL: http://arxiv.org/abs/2411.07899v2
• Date: Mon, 18 Nov 2024 07:13:24 GMT
• Title: Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer
• Authors: Xiao Huo, Junhui Hou, Shuai Wan, Fuzheng Yang,
• Abstract summary: 3D visualization techniques have fundamentally transformed how we interact with digital content. Massive data size of point clouds presents significant challenges in data compression. We propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering.
• Score: 52.40992954884257
• License:
• Abstract: The evolution of 3D visualization techniques has fundamentally transformed how we interact with digital content. At the forefront of this change is point cloud technology, offering an immersive experience that surpasses traditional 2D representations. However, the massive data size of point clouds presents significant challenges in data compression. Current methods for lossy point cloud attribute compression (PCAC) generally focus on reconstructing the original point clouds with minimal error. However, for point cloud visualization scenarios, the reconstructed point clouds with distortion still need to undergo a complex rendering process, which affects the final user-perceived quality. In this paper, we propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering, denoted as rendering-oriented PCAC (RO-PCAC), directly targeting the quality of rendered multiview images for viewing. In a differentiable manner, the impact of the rendering process on the reconstructed point clouds is taken into account. Moreover, we characterize point clouds as sparse tensors and propose a sparse tensor-based transformer, called SP-Trans. By aligning with the local density of the point cloud and utilizing an enhanced local attention mechanism, SP-Trans captures the intricate relationships within the point cloud, further improving feature analysis and synthesis within the framework. Extensive experiments demonstrate that the proposed RO-PCAC achieves state-of-the-art compression performance, compared to existing reconstruction-oriented methods, including traditional, learning-based, and hybrid methods.

Related papers

• Point Cloud Compression with Implicit Neural Representations: A Unified Framework [54.119415852585306]
  We present a pioneering point cloud compression framework capable of handling both geometry and attribute components. Our framework utilizes two coordinate-based neural networks to implicitly represent a voxelized point cloud. Our method exhibits high universality when contrasted with existing learning-based techniques.
  arXiv  Detail & Related papers  (2024-05-19T09:19:40Z)
• PIVOT-Net: Heterogeneous Point-Voxel-Tree-based Framework for Point Cloud Compression [8.778300313732027]
  We propose a heterogeneous point cloud compression (PCC) framework. We unify typical point cloud representations -- point-based, voxel-based, and tree-based representations -- and their associated backbones. We augment the framework with a proposed context-aware upsampling for decoding and an enhanced voxel transformer for feature aggregation.
  arXiv  Detail & Related papers  (2024-02-11T16:57:08Z)
• Geometric Prior Based Deep Human Point Cloud Geometry Compression [67.49785946369055]
  We leverage the human geometric prior in geometry redundancy removal of point clouds. We can envisage high-resolution human point clouds as a combination of geometric priors and structural deviations. The proposed framework can operate in a play-and-plug fashion with existing learning based point cloud compression methods.
  arXiv  Detail & Related papers  (2023-05-02T10:35:20Z)
• GRASP-Net: Geometric Residual Analysis and Synthesis for Point Cloud Compression [16.98171403698783]
  We propose a heterogeneous approach with deep learning for lossy point cloud geometry compression. Specifically, a point-based network is applied to convert the erratic local details to latent features residing on the coarse point cloud.
  arXiv  Detail & Related papers  (2022-09-09T17:09:02Z)
• IPDAE: Improved Patch-Based Deep Autoencoder for Lossy Point Cloud Geometry Compression [11.410441760314564]
  We propose a set of significant improvements to patch-based point cloud compression. Experiments show that the improved patch-based autoencoder outperforms the state-of-the-art in terms of rate-distortion performance.
  arXiv  Detail & Related papers  (2022-08-04T08:12:35Z)
• Variable Rate Compression for Raw 3D Point Clouds [5.107705550575662]
  We propose a novel variable rate deep compression architecture that operates on raw 3D point cloud data. Our network is capable of explicitly processing point clouds and generating a compressed description.
  arXiv  Detail & Related papers  (2022-02-28T15:15:39Z)
• PU-Transformer: Point Cloud Upsampling Transformer [38.05362492645094]
  We focus on the point cloud upsampling task that intends to generate dense high-fidelity point clouds from sparse input data. Specifically, to activate the transformer's strong capability in representing features, we develop a new variant of a multi-head self-attention structure. We demonstrate the outstanding performance of our approach by comparing with the state-of-the-art CNN-based methods on different benchmarks.
  arXiv  Detail & Related papers  (2021-11-24T03:25:35Z)
• Deep Point Cloud Reconstruction [74.694733918351]
  Point cloud obtained from 3D scanning is often sparse, noisy, and irregular. To cope with these issues, recent studies have been separately conducted to densify, denoise, and complete inaccurate point cloud. We propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse stacked-hourglass network as for the initial densification and denoising, 2) a refinement via transformers converting the discrete voxels into 3D points.
  arXiv  Detail & Related papers  (2021-11-23T07:53:28Z)
• PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers [81.71904691925428]
  We present a new method that reformulates point cloud completion as a set-to-set translation problem. We also design a new model, called PoinTr, that adopts a transformer encoder-decoder architecture for point cloud completion. Our method outperforms state-of-the-art methods by a large margin on both the new benchmarks and the existing ones.
  arXiv  Detail & Related papers  (2021-08-19T17:58:56Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.