Deeply Learned Spectral Total Variation Decomposition
- URL: http://arxiv.org/abs/2006.10004v2
- Date: Wed, 21 Oct 2020 17:03:54 GMT
- Title: Deeply Learned Spectral Total Variation Decomposition
- Authors: Tamara G. Grossmann, Yury Korolev, Guy Gilboa, Carola-Bibiane
Sch\"onlieb
- Abstract summary: We present a neural network approximation of a non-linear spectral decomposition.
We report up to four orders of magnitude ($times 10,000$) speedup in processing of mega-pixel size images.
- Score: 8.679020335206753
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Non-linear spectral decompositions of images based on one-homogeneous
functionals such as total variation have gained considerable attention in the
last few years. Due to their ability to extract spectral components
corresponding to objects of different size and contrast, such decompositions
enable filtering, feature transfer, image fusion and other applications.
However, obtaining this decomposition involves solving multiple non-smooth
optimisation problems and is therefore computationally highly intensive. In
this paper, we present a neural network approximation of a non-linear spectral
decomposition. We report up to four orders of magnitude ($\times 10,000$)
speedup in processing of mega-pixel size images, compared to classical GPU
implementations. Our proposed network, TVSpecNET, is able to implicitly learn
the underlying PDE and, despite being entirely data driven, inherits
invariances of the model based transform. To the best of our knowledge, this is
the first approach towards learning a non-linear spectral decomposition of
images. Not only do we gain a staggering computational advantage, but this
approach can also be seen as a step towards studying neural networks that can
decompose an image into spectral components defined by a user rather than a
handcrafted functional.
Related papers
- Coarse-Fine Spectral-Aware Deformable Convolution For Hyperspectral Image Reconstruction [15.537910100051866]
We study the inverse problem of Coded Aperture Snapshot Spectral Imaging (CASSI)
We propose Coarse-Fine Spectral-Aware Deformable Convolution Network (CFSDCN)
Our CFSDCN significantly outperforms previous state-of-the-art (SOTA) methods on both simulated and real HSI datasets.
arXiv Detail & Related papers (2024-06-18T15:15:12Z) - Discriminative feature encoding for intrinsic image decomposition [16.77439691640257]
Intrinsic image decomposition is an important and long-standing computer vision problem.
This work takes advantage of deep learning, and shows that it can solve this challenging computer vision problem with high efficiency.
arXiv Detail & Related papers (2022-09-25T05:51:49Z) - Orthonormal Convolutions for the Rotation Based Iterative
Gaussianization [64.44661342486434]
This paper elaborates an extension of rotation-based iterative Gaussianization, RBIG, which makes image Gaussianization possible.
In images its application has been restricted to small image patches or isolated pixels, because rotation in RBIG is based on principal or independent component analysis.
We present the emphConvolutional RBIG: an extension that alleviates this issue by imposing that the rotation in RBIG is a convolution.
arXiv Detail & Related papers (2022-06-08T12:56:34Z) - An Optical physics inspired CNN approach for intrinsic image
decomposition [0.0]
Intrinsic Image Decomposition is an open problem of generating the constituents of an image.
We propose a neural network architecture capable of this decomposition using physics-based parameters derived from the image.
arXiv Detail & Related papers (2021-05-21T00:54:01Z) - Efficient and Differentiable Shadow Computation for Inverse Problems [64.70468076488419]
Differentiable geometric computation has received increasing interest for image-based inverse problems.
We propose an efficient yet efficient approach for differentiable visibility and soft shadow computation.
As our formulation is differentiable, it can be used to solve inverse problems such as texture, illumination, rigid pose, and deformation recovery from images.
arXiv Detail & Related papers (2021-04-01T09:29:05Z) - Regularization by Denoising Sub-sampled Newton Method for Spectral CT
Multi-Material Decomposition [78.37855832568569]
We propose to solve a model-based maximum-a-posterior problem to reconstruct multi-materials images with application to spectral CT.
In particular, we propose to solve a regularized optimization problem based on a plug-in image-denoising function.
We show numerical and experimental results for spectral CT materials decomposition.
arXiv Detail & Related papers (2021-03-25T15:20:10Z) - A Loss Function for Generative Neural Networks Based on Watson's
Perceptual Model [14.1081872409308]
To train Variational Autoencoders (VAEs) to generate realistic imagery requires a loss function that reflects human perception of image similarity.
We propose such a loss function based on Watson's perceptual model, which computes a weighted distance in frequency space and accounts for luminance and contrast masking.
In experiments, VAEs trained with the new loss function generated realistic, high-quality image samples.
arXiv Detail & Related papers (2020-06-26T15:36:11Z) - A Flexible Framework for Designing Trainable Priors with Adaptive
Smoothing and Game Encoding [57.1077544780653]
We introduce a general framework for designing and training neural network layers whose forward passes can be interpreted as solving non-smooth convex optimization problems.
We focus on convex games, solved by local agents represented by the nodes of a graph and interacting through regularization functions.
This approach is appealing for solving imaging problems, as it allows the use of classical image priors within deep models that are trainable end to end.
arXiv Detail & Related papers (2020-06-26T08:34:54Z) - Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral
Imagery [79.69449412334188]
In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches.
We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data.
Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
arXiv Detail & Related papers (2020-05-18T14:25:50Z) - Spectral Learning on Matrices and Tensors [74.88243719463053]
We show that tensor decomposition can pick up latent effects that are missed by matrix methods.
We also outline computational techniques to design efficient tensor decomposition methods.
arXiv Detail & Related papers (2020-04-16T22:53:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.