GPA-Net:No-Reference Point Cloud Quality Assessment with Multi-task Graph Convolutional Network

• URL: http://arxiv.org/abs/2210.16478v3
• Date: Thu, 1 Jun 2023 14:42:23 GMT
• Title: GPA-Net:No-Reference Point Cloud Quality Assessment with Multi-task Graph Convolutional Network
• Authors: Ziyu Shan, Qi Yang, Rui Ye, Yujie Zhang, Yiling Xu, Xiaozhong Xu and Shan Liu
• Abstract summary: We propose a novel no-reference PCQA metric named the Graph convolutional PCQA network (GPA-Net) To extract effective features for PCQA, we propose a new graph convolution kernel, i.e., GPAConv, which attentively captures the perturbation of structure and texture. Experimental results on two independent databases show that GPA-Net achieves the best performance compared to the state-of-the-art no-reference PCQA metrics.
• Score: 35.381247959766505
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rapid development of 3D vision, point cloud has become an increasingly popular 3D visual media content. Due to the irregular structure, point cloud has posed novel challenges to the related research, such as compression, transmission, rendering and quality assessment. In these latest researches, point cloud quality assessment (PCQA) has attracted wide attention due to its significant role in guiding practical applications, especially in many cases where the reference point cloud is unavailable. However, current no-reference metrics which based on prevalent deep neural network have apparent disadvantages. For example, to adapt to the irregular structure of point cloud, they require preprocessing such as voxelization and projection that introduce extra distortions, and the applied grid-kernel networks, such as Convolutional Neural Networks, fail to extract effective distortion-related features. Besides, they rarely consider the various distortion patterns and the philosophy that PCQA should exhibit shifting, scaling, and rotational invariance. In this paper, we propose a novel no-reference PCQA metric named the Graph convolutional PCQA network (GPA-Net). To extract effective features for PCQA, we propose a new graph convolution kernel, i.e., GPAConv, which attentively captures the perturbation of structure and texture. Then, we propose the multi-task framework consisting of one main task (quality regression) and two auxiliary tasks (distortion type and degree predictions). Finally, we propose a coordinate normalization module to stabilize the results of GPAConv under shift, scale and rotation transformations. Experimental results on two independent databases show that GPA-Net achieves the best performance compared to the state-of-the-art no-reference PCQA metrics, even better than some full-reference metrics in some cases.

Related papers

• No-Reference Point Cloud Quality Assessment via Graph Convolutional Network [89.12589881881082]
  Three-dimensional (3D) point cloud, as an emerging visual media format, is increasingly favored by consumers. Point clouds inevitably suffer from quality degradation and information loss through multimedia communication systems. We propose a novel no-reference PCQA method by using a graph convolutional network (GCN) to characterize the mutual dependencies of multi-view 2D projected image contents.
  arXiv  Detail & Related papers  (2024-11-12T11:39:05Z)
• PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learning [56.14518823931901]
  We present PointRWKV, a model of linear complexity derived from the RWKV model in the NLP field. We first propose to explore the global processing capabilities within PointRWKV blocks using modified multi-headed matrix-valued states. To extract local geometric features simultaneously, we design a parallel branch to encode the point cloud efficiently in a fixed radius near-neighbors graph with a graph stabilizer.
  arXiv  Detail & Related papers  (2024-05-24T05:02:51Z)
• Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment [49.36799270585947]
  No-reference point cloud quality assessment (NR-PCQA) aims to automatically evaluate the perceptual quality of distorted point clouds without available reference. We propose a novel contrastive pre-training framework tailored for PCQA (CoPA) Our method outperforms the state-of-the-art PCQA methods on popular benchmarks.
  arXiv  Detail & Related papers  (2024-03-15T07:16:07Z)
• Activating Frequency and ViT for 3D Point Cloud Quality Assessment without Reference [0.49157446832511503]
  We propose no-reference quality metric of a given 3D-PC. To map the input attributes to quality score, we use a light-weight hybrid deep model; combined of Deformable Convolutional Network (DCN) and Vision Transformers (ViT) The results show that our approach outperforms state-of-the-art NR-PCQA measures and even some FR-PCQA on PointXR.
  arXiv  Detail & Related papers  (2023-12-10T19:13:34Z)
• Simple Baselines for Projection-based Full-reference and No-reference Point Cloud Quality Assessment [60.2709006613171]
  We propose simple baselines for projection-based point cloud quality assessment (PCQA) We use multi-projections obtained via a common cube-like projection process from the point clouds for both full-reference (FR) and no-reference (NR) PCQA tasks. Taking part in the ICIP 2023 PCVQA Challenge, we succeeded in achieving the top spot in four out of the five competition tracks.
  arXiv  Detail & Related papers  (2023-10-26T04:42:57Z)
• TCDM: Transformational Complexity Based Distortion Metric for Perceptual Point Cloud Quality Assessment [24.936061591860838]
  The goal of objective point cloud quality assessment (PCQA) research is to develop metrics that measure point cloud quality in a consistent manner. We evaluate the point cloud quality by measuring the complexity of transforming the distorted point cloud back to its reference. The effectiveness of the proposed transformational complexity based distortion metric (TCDM) is evaluated through extensive experiments conducted on five public point cloud quality assessment databases.
  arXiv  Detail & Related papers  (2022-10-10T13:20:51Z)
• MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment [32.495387943305204]
  We propose a novel no-reference point cloud quality assessment (NR-PCQA) metric in a multi-modal fashion. In specific, we split the point clouds into sub-models to represent local geometry distortions such as point shift and down-sampling. To achieve the goals, the sub-models and projected images are encoded with point-based and image-based neural networks.
  arXiv  Detail & Related papers  (2022-09-01T06:11:12Z)
• No-Reference Point Cloud Quality Assessment via Domain Adaptation [31.280188860021248]
  We present a novel no-reference quality assessment metric, the image transferred point cloud quality assessment (IT-PCQA) for 3D point clouds. In particular, we treat natural images as the source domain and point clouds as the target domain, and infer point cloud quality via unsupervised adversarial domain adaptation. Experimental results show that the proposed method can achieve higher performance than traditional no-reference metrics, even comparable results with full-reference metrics.
  arXiv  Detail & Related papers  (2021-12-06T08:20:40Z)
• PC-RGNN: Point Cloud Completion and Graph Neural Network for 3D Object Detection [57.49788100647103]
  LiDAR-based 3D object detection is an important task for autonomous driving. Current approaches suffer from sparse and partial point clouds of distant and occluded objects. In this paper, we propose a novel two-stage approach, namely PC-RGNN, dealing with such challenges by two specific solutions.
  arXiv  Detail & Related papers  (2020-12-18T18:06:43Z)
• Learning Deep Interleaved Networks with Asymmetric Co-Attention for Image Restoration [65.11022516031463]
  We present a deep interleaved network (DIN) that learns how information at different states should be combined for high-quality (HQ) images reconstruction. In this paper, we propose asymmetric co-attention (AsyCA) which is attached at each interleaved node to model the feature dependencies. Our presented DIN can be trained end-to-end and applied to various image restoration tasks.
  arXiv  Detail & Related papers  (2020-10-29T15:32:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.