NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods

• URL: http://arxiv.org/abs/2412.08029v1
• Date: Wed, 11 Dec 2024 02:17:33 GMT
• Title: NeRF-NQA: No-Reference Quality Assessment for Scenes Generated by NeRF and Neural View Synthesis Methods
• Authors: Qiang Qu, Hanxue Liang, Xiaoming Chen, Yuk Ying Chung, Yiran Shen,
• Abstract summary: We propose NeRF-NQA, the first no-reference quality assessment method for densely-observed scenes synthesized from the NVS and NeRF variants. NeRF-NQA employs a joint quality assessment strategy, integrating both viewwise and pointwise approaches. The viewwise approach assesses the spatial quality of each individual synthesized view and the overall inter-views consistency, while the pointwise approach focuses on the angular qualities of scene surface points.
• Score: 13.403739247879766
• License:
• Abstract: Neural View Synthesis (NVS) has demonstrated efficacy in generating high-fidelity dense viewpoint videos using a image set with sparse views. However, existing quality assessment methods like PSNR, SSIM, and LPIPS are not tailored for the scenes with dense viewpoints synthesized by NVS and NeRF variants, thus, they often fall short in capturing the perceptual quality, including spatial and angular aspects of NVS-synthesized scenes. Furthermore, the lack of dense ground truth views makes the full reference quality assessment on NVS-synthesized scenes challenging. For instance, datasets such as LLFF provide only sparse images, insufficient for complete full-reference assessments. To address the issues above, we propose NeRF-NQA, the first no-reference quality assessment method for densely-observed scenes synthesized from the NVS and NeRF variants. NeRF-NQA employs a joint quality assessment strategy, integrating both viewwise and pointwise approaches, to evaluate the quality of NVS-generated scenes. The viewwise approach assesses the spatial quality of each individual synthesized view and the overall inter-views consistency, while the pointwise approach focuses on the angular qualities of scene surface points and their compound inter-point quality. Extensive evaluations are conducted to compare NeRF-NQA with 23 mainstream visual quality assessment methods (from fields of image, video, and light-field assessment). The results demonstrate NeRF-NQA outperforms the existing assessment methods significantly and it shows substantial superiority on assessing NVS-synthesized scenes without references. An implementation of this paper are available at https://github.com/VincentQQu/NeRF-NQA.

Related papers

• GS-QA: Comprehensive Quality Assessment Benchmark for Gaussian Splatting View Synthesis [4.117347527143616]
  Gaussian Splatting (GS) offers a promising alternative to Neural Radiance Fields (NeRF) for real-time 3D scene rendering. GS achieves faster rendering times and reduced memory consumption compared to the neural network approach used in NeRF. This paper describes a subjective quality assessment study that aims to evaluate synthesized videos obtained with several static GS methods.
  arXiv  Detail & Related papers  (2025-02-18T17:46:57Z)
• Evaluating Human Perception of Novel View Synthesis: Subjective Quality Assessment of Gaussian Splatting and NeRF in Dynamic Scenes [6.157597876333952]
  We conduct two subjective experiments for the quality assessment of NVS technologies containing both GS-based and NeRF-based methods. This study covers 360deg, front-facing, and single-viewpoint photorealistic videos while providing a richer and greater number of real scenes. It's the first time to explore the impact of NVS methods in dynamic scenes with moving objects.
  arXiv  Detail & Related papers  (2025-01-13T10:01:27Z)
• NVS-SQA: Exploring Self-Supervised Quality Representation Learning for Neurally Synthesized Scenes without References [55.35182166250742]
  We propose NVS-SQA, a quality assessment method to learn no-reference quality representations through self-supervision. Traditional self-supervised learning predominantly relies on the "same instance, similar representation" assumption and extensive datasets. We employ photorealistic cues and quality scores as learning objectives, along with a specialized contrastive pair preparation process to improve the effectiveness and efficiency of learning.
  arXiv  Detail & Related papers  (2025-01-11T09:12:43Z)
• Adaptive Feature Selection for No-Reference Image Quality Assessment by Mitigating Semantic Noise Sensitivity [55.399230250413986]
  We propose a Quality-Aware Feature Matching IQA Metric (QFM-IQM) to remove harmful semantic noise features from the upstream task. Our approach achieves superior performance to the state-of-the-art NR-IQA methods on eight standard IQA datasets.
  arXiv  Detail & Related papers  (2023-12-11T06:50:27Z)
• Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front-Facing Views [10.565297375544414]
  We present the first study on perceptual evaluation of NVS and NeRF variants. We measured the quality of videos synthesized by several NVS methods in a well-controlled perceptual quality assessment experiment.
  arXiv  Detail & Related papers  (2023-03-24T11:53:48Z)
• Reduced-Reference Quality Assessment of Point Clouds via Content-Oriented Saliency Projection [17.983188216548005]
  Many dense 3D point clouds have been exploited to represent visual objects instead of traditional images or videos. We propose a novel and efficient Reduced-Reference quality metric for point clouds.
  arXiv  Detail & Related papers  (2023-01-18T18:00:29Z)
• Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment [60.57703721744873]
  The increased resolution of real-world videos presents a dilemma between efficiency and accuracy for deep Video Quality Assessment (VQA) In this work, we propose a unified scheme, spatial-temporal grid mini-cube sampling (St-GMS) to get a novel type of sample, named fragments. With fragments and FANet, the proposed efficient end-to-end FAST-VQA and FasterVQA achieve significantly better performance than existing approaches on all VQA benchmarks.
  arXiv  Detail & Related papers  (2022-10-11T11:38:07Z)
• SiNeRF: Sinusoidal Neural Radiance Fields for Joint Pose Estimation and Scene Reconstruction [147.9379707578091]
  NeRFmm is the Neural Radiance Fields (NeRF) that deal with Joint Optimization tasks. Despite NeRFmm producing precise scene synthesis and pose estimations, it still struggles to outperform the full-annotated baseline on challenging scenes. We propose Sinusoidal Neural Radiance Fields (SiNeRF) that leverage sinusoidal activations for radiance mapping and a novel Mixed Region Sampling (MRS) for selecting ray batch efficiently.
  arXiv  Detail & Related papers  (2022-10-10T10:47:51Z)
• Learning Transformer Features for Image Quality Assessment [53.51379676690971]
  We propose a unified IQA framework that utilizes CNN backbone and transformer encoder to extract features. The proposed framework is compatible with both FR and NR modes and allows for a joint training scheme.
  arXiv  Detail & Related papers  (2021-12-01T13:23:00Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.