Deeply Learned Spectral Total Variation Decomposition

• URL: http://arxiv.org/abs/2006.10004v2
• Date: Wed, 21 Oct 2020 17:03:54 GMT
• Title: Deeply Learned Spectral Total Variation Decomposition
• Authors: Tamara G. Grossmann, Yury Korolev, Guy Gilboa, Carola-Bibiane Sch\"onlieb
• Abstract summary: We present a neural network approximation of a non-linear spectral decomposition. We report up to four orders of magnitude ($times 10,000$) speedup in processing of mega-pixel size images.
• Score: 8.679020335206753
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Non-linear spectral decompositions of images based on one-homogeneous functionals such as total variation have gained considerable attention in the last few years. Due to their ability to extract spectral components corresponding to objects of different size and contrast, such decompositions enable filtering, feature transfer, image fusion and other applications. However, obtaining this decomposition involves solving multiple non-smooth optimisation problems and is therefore computationally highly intensive. In this paper, we present a neural network approximation of a non-linear spectral decomposition. We report up to four orders of magnitude ($\times 10,000$) speedup in processing of mega-pixel size images, compared to classical GPU implementations. Our proposed network, TVSpecNET, is able to implicitly learn the underlying PDE and, despite being entirely data driven, inherits invariances of the model based transform. To the best of our knowledge, this is the first approach towards learning a non-linear spectral decomposition of images. Not only do we gain a staggering computational advantage, but this approach can also be seen as a step towards studying neural networks that can decompose an image into spectral components defined by a user rather than a handcrafted functional.

Related papers

• Multispectral Texture Synthesis using RGB Convolutional Neural Networks [2.3213238782019316]
  State-of-the-art RGB texture synthesis algorithms rely on style distances that are computed through statistics of deep features. We propose two solutions to extend these methods to multispectral imaging.
  arXiv  Detail & Related papers  (2024-10-21T13:49:54Z)
• Neural Spectral Decomposition for Dataset Distillation [48.59372086450124]
  We propose Neural Spectrum Decomposition, a generic decomposition framework for dataset distillation. We aim to discover the low-rank representation of the entire dataset and perform distillation efficiently. Our results demonstrate that our approach achieves state-of-the-art performance on benchmarks, including CIFAR10, CIFAR100, Tiny Imagenet, and ImageNet Subset.
  arXiv  Detail & Related papers  (2024-08-29T03:26:14Z)
• Deep Learning Based Speckle Filtering for Polarimetric SAR Images. Application to Sentinel-1 [51.404644401997736]
  We propose a complete framework to remove speckle in polarimetric SAR images using a convolutional neural network. Experiments show that the proposed approach offers exceptional results in both speckle reduction and resolution preservation.
  arXiv  Detail & Related papers  (2024-08-28T10:07:17Z)
• An Optical physics inspired CNN approach for intrinsic image decomposition [0.0]
  Intrinsic Image Decomposition is an open problem of generating the constituents of an image. We propose a neural network architecture capable of this decomposition using physics-based parameters derived from the image.
  arXiv  Detail & Related papers  (2021-05-21T00:54:01Z)
• Efficient and Differentiable Shadow Computation for Inverse Problems [64.70468076488419]
  Differentiable geometric computation has received increasing interest for image-based inverse problems. We propose an efficient yet efficient approach for differentiable visibility and soft shadow computation. As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and deformation recovery from images.
  arXiv  Detail & Related papers  (2021-04-01T09:29:05Z)
• Regularization by Denoising Sub-sampled Newton Method for Spectral CT Multi-Material Decomposition [78.37855832568569]
  We propose to solve a model-based maximum-a-posterior problem to reconstruct multi-materials images with application to spectral CT. In particular, we propose to solve a regularized optimization problem based on a plug-in image-denoising function. We show numerical and experimental results for spectral CT materials decomposition.
  arXiv  Detail & Related papers  (2021-03-25T15:20:10Z)
• A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model [14.1081872409308]
  To train Variational Autoencoders (VAEs) to generate realistic imagery requires a loss function that reflects human perception of image similarity. We propose such a loss function based on Watson's perceptual model, which computes a weighted distance in frequency space and accounts for luminance and contrast masking. In experiments, VAEs trained with the new loss function generated realistic, high-quality image samples.
  arXiv  Detail & Related papers  (2020-06-26T15:36:11Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)
• Spectral Learning on Matrices and Tensors [74.88243719463053]
  We show that tensor decomposition can pick up latent effects that are missed by matrix methods. We also outline computational techniques to design efficient tensor decomposition methods.
  arXiv  Detail & Related papers  (2020-04-16T22:53:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.