RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network

• URL: http://arxiv.org/abs/2404.07766v2
• Date: Sun, 14 Apr 2024 13:14:54 GMT
• Title: RMAFF-PSN: A Residual Multi-Scale Attention Feature Fusion Photometric Stereo Network
• Authors: Kai Luo, Yakun Ju, Lin Qi, Kaixuan Wang, Junyu Dong,
• Abstract summary: Predicting accurate maps of objects from two-dimensional images in regions of complex structure spatial material variations is challenging. We propose a method of calibrated feature information from different resolution stages and scales of the image. This approach preserves more physical information, such as texture and geometry of the object in complex regions.
• Score: 37.759675702107586
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predicting accurate normal maps of objects from two-dimensional images in regions of complex structure and spatial material variations is challenging using photometric stereo methods due to the influence of surface reflection properties caused by variations in object geometry and surface materials. To address this issue, we propose a photometric stereo network called a RMAFF-PSN that uses residual multiscale attentional feature fusion to handle the ``difficult'' regions of the object. Unlike previous approaches that only use stacked convolutional layers to extract deep features from the input image, our method integrates feature information from different resolution stages and scales of the image. This approach preserves more physical information, such as texture and geometry of the object in complex regions, through shallow-deep stage feature extraction, double branching enhancement, and attention optimization. To test the network structure under real-world conditions, we propose a new real dataset called Simple PS data, which contains multiple objects with varying structures and materials. Experimental results on a publicly available benchmark dataset demonstrate that our method outperforms most existing calibrated photometric stereo methods for the same number of input images, especially in the case of highly non-convex object structures. Our method also obtains good results under sparse lighting conditions.

Related papers

• Deep Learning Methods for Calibrated Photometric Stereo and Beyond [86.57469194387264]
  Photometric stereo recovers the surface normals of an object from multiple images with varying shading cues. Deep learning methods have shown a powerful ability in the context of photometric stereo against non-Lambertian surfaces.
  arXiv  Detail & Related papers  (2022-12-16T11:27:44Z)
• MS-PS: A Multi-Scale Network for Photometric Stereo With a New Comprehensive Training Dataset [0.0]
  Photometric stereo (PS) problem consists in reconstructing the 3D-surface of an object. We propose a multi-scale architecture for PS which, combined with a new dataset, yields state-of-the-art results.
  arXiv  Detail & Related papers  (2022-11-25T14:01:54Z)
• Deep Uncalibrated Photometric Stereo via Inter-Intra Image Feature Fusion [17.686973510425172]
  This paper presents a new method for deep uncalibrated photometric stereo. It efficiently utilizes the inter-image representation to guide the normal estimation. Our method produces significantly better results than the state-of-the-art methods on both synthetic and real data.
  arXiv  Detail & Related papers  (2022-08-06T03:59:54Z)
• Uncertainty-Aware Deep Multi-View Photometric Stereo [100.97116470055273]
  Photometric stereo (PS) is excellent at recovering high-frequency surface details, whereas multi-view stereo (MVS) can help remove the low-frequency distortion due to PS and retain the global shape. This paper proposes an approach that can effectively utilize such complementary strengths of PS and MVS. We estimate per-pixel surface normals and depth using an uncertainty-aware deep-PS network and deep-MVS network, respectively.
  arXiv  Detail & Related papers  (2022-02-26T05:45:52Z)
• NeROIC: Neural Rendering of Objects from Online Image Collections [42.02832046768925]
  We present a novel method to acquire object representations from online image collections, capturing high-quality geometry and material properties of arbitrary objects. This enables various object-centric rendering applications such as novel-view synthesis, relighting, and harmonized background composition.
  arXiv  Detail & Related papers  (2022-01-07T16:45:15Z)
• Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo [103.08512487830669]
  We present a modern solution to the multi-view photometric stereo problem (MVPS) We procure the surface orientation using a photometric stereo (PS) image formation model and blend it with a multi-view neural radiance field representation to recover the object's surface geometry. Our method performs neural rendering of multi-view images while utilizing surface normals estimated by a deep photometric stereo network.
  arXiv  Detail & Related papers  (2021-10-11T20:20:03Z)
• Leveraging Spatial and Photometric Context for Calibrated Non-Lambertian Photometric Stereo [61.6260594326246]
  We introduce an efficient fully-convolutional architecture that can leverage both spatial and photometric context simultaneously. Using separable 4D convolutions and 2D heat-maps reduces the size and makes more efficient.
  arXiv  Detail & Related papers  (2021-03-22T18:06:58Z)
• Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo [14.5172791293107]
  We build a two-stage Convolutional Neural Network (CNN) architecture to construct inter- and intra-frame representations. We experimentally investigate numerous network design alternatives for identifying the optimal scheme to deploy inter-frame and intra-frame feature extraction modules.
  arXiv  Detail & Related papers  (2020-12-26T11:22:56Z)
• Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Images [59.906948203578544]
  We introduce a novel learning-based method to reconstruct the high-quality geometry and complex, spatially-varying BRDF of an arbitrary object. We first estimate per-view depth maps using a deep multi-view stereo network. These depth maps are used to coarsely align the different views. We propose a novel multi-view reflectance estimation network architecture.
  arXiv  Detail & Related papers  (2020-03-27T21:28:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.