Towards virtual painting recolouring using Vision Transformer on X-Ray Fluorescence datacubes

• URL: http://arxiv.org/abs/2410.08826v1
• Date: Fri, 11 Oct 2024 14:05:28 GMT
• Title: Towards virtual painting recolouring using Vision Transformer on X-Ray Fluorescence datacubes
• Authors: Alessandro Bombini, Fernando García-Avello Bofías, Francesca Giambi, Chiara Ruberto,
• Abstract summary: We define a pipeline to perform virtual painting recolouring using raw data of X-Ray Fluorescence (XRF) analysis on pictorial artworks. To circumvent the small dataset size, we generate a synthetic dataset, starting from a database of XRF spectra. We define a Deep Variational Embedding network to embed the XRF spectra into a lower dimensional, K-Means friendly, metric space.
• Score: 80.32085982862151
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this contribution, we define (and test) a pipeline to perform virtual painting recolouring using raw data of X-Ray Fluorescence (XRF) analysis on pictorial artworks. To circumvent the small dataset size, we generate a synthetic dataset, starting from a database of XRF spectra; furthermore, to ensure a better generalisation capacity (and to tackle the issue of in-memory size and inference time), we define a Deep Variational Embedding network to embed the XRF spectra into a lower dimensional, K-Means friendly, metric space. We thus train a set of models to assign coloured images to embedded XRF images. We report here the devised pipeline performances in terms of visual quality metrics, and we close on a discussion on the results.

Related papers

• MatSpectNet: Material Segmentation Network with Domain-Aware and Physically-Constrained Hyperspectral Reconstruction [13.451692195639696]
  MatSpectNet is a new model to segment materials with recovered hyperspectral images from RGB images. It exploits the principles of colour perception in modern cameras to constrain the reconstructed hyperspectral images. It attains a 1.60% increase in average pixel accuracy and a 3.42% improvement in mean class accuracy compared with the most recent publication.
  arXiv  Detail & Related papers  (2023-07-21T10:02:02Z)
• Multiscale Representation for Real-Time Anti-Aliasing Neural Rendering [84.37776381343662]
  Mip-NeRF proposes a multiscale representation as a conical frustum to encode scale information. We propose mip voxel grids (Mip-VoG), an explicit multiscale representation for real-time anti-aliasing rendering. Our approach is the first to offer multiscale training and real-time anti-aliasing rendering simultaneously.
  arXiv  Detail & Related papers  (2023-04-20T04:05:22Z)
• RTMV: A Ray-Traced Multi-View Synthetic Dataset for Novel View Synthesis [104.53930611219654]
  We present a large-scale synthetic dataset for novel view synthesis consisting of 300k images rendered from nearly 2000 complex scenes. The dataset is orders of magnitude larger than existing synthetic datasets for novel view synthesis. Using 4 distinct sources of high-quality 3D meshes, the scenes of our dataset exhibit challenging variations in camera views, lighting, shape, materials, and textures.
  arXiv  Detail & Related papers  (2022-05-14T13:15:32Z)
• NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling [82.99453001445478]
  We present NeRF-SR, a solution for high-resolution (HR) novel view synthesis with mostly low-resolution (LR) inputs. Our method is built upon Neural Radiance Fields (NeRF) that predicts per-point density and color with a multi-layer perceptron.
  arXiv  Detail & Related papers  (2021-12-03T07:33:47Z)
• Learning Neural Light Fields with Ray-Space Embedding Networks [51.88457861982689]
  We propose a novel neural light field representation that is compact and directly predicts integrated radiance along rays. Our method achieves state-of-the-art quality on dense forward-facing datasets such as the Stanford Light Field dataset.
  arXiv  Detail & Related papers  (2021-12-02T18:59:51Z)
• iNeRF: Inverting Neural Radiance Fields for Pose Estimation [68.91325516370013]
  We present iNeRF, a framework that performs mesh-free pose estimation by "inverting" a Neural RadianceField (NeRF) NeRFs have been shown to be remarkably effective for the task of view synthesis.
  arXiv  Detail & Related papers  (2020-12-10T18:36:40Z)
• Image inpainting using frequency domain priors [35.54138025375951]
  We present a novel image inpainting technique using frequency domain information. We evaluate our proposed method on the publicly available datasets CelebA, Paris Streetview, and DTD texture dataset.
  arXiv  Detail & Related papers  (2020-12-03T11:08:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.