Generic Perceptual Loss for Modeling Structured Output Dependencies
- URL: http://arxiv.org/abs/2103.10571v1
- Date: Thu, 18 Mar 2021 23:56:07 GMT
- Title: Generic Perceptual Loss for Modeling Structured Output Dependencies
- Authors: Yifan Liu, Hao Chen, Yu Chen, Wei Yin, Chunhua Shen
- Abstract summary: We show that, what matters is the network structure instead of the trained weights.
We demonstrate that a randomly-weighted deep CNN can be used to model the structured dependencies of outputs.
- Score: 78.59700528239141
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: The perceptual loss has been widely used as an effective loss term in image
synthesis tasks including image super-resolution, and style transfer. It was
believed that the success lies in the high-level perceptual feature
representations extracted from CNNs pretrained with a large set of images. Here
we reveal that, what matters is the network structure instead of the trained
weights. Without any learning, the structure of a deep network is sufficient to
capture the dependencies between multiple levels of variable statistics using
multiple layers of CNNs. This insight removes the requirements of pre-training
and a particular network structure (commonly, VGG) that are previously assumed
for the perceptual loss, thus enabling a significantly wider range of
applications. To this end, we demonstrate that a randomly-weighted deep CNN can
be used to model the structured dependencies of outputs. On a few dense
per-pixel prediction tasks such as semantic segmentation, depth estimation and
instance segmentation, we show improved results of using the extended
randomized perceptual loss, compared to the baselines using pixel-wise loss
alone. We hope that this simple, extended perceptual loss may serve as a
generic structured-output loss that is applicable to most structured output
learning tasks.
Related papers
- Distance Weighted Trans Network for Image Completion [52.318730994423106]
We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components.
CNNs are used to augment the local texture information of coarse priors.
DWT blocks are used to recover certain coarse textures and coherent visual structures.
arXiv Detail & Related papers (2023-10-11T12:46:11Z) - FourierLoss: Shape-Aware Loss Function with Fourier Descriptors [1.5659201748872393]
This work introduces a new shape-aware loss function, which we name FourierLoss.
It relies on the shape dissimilarity between the ground truth and the predicted segmentation maps through the Fourier descriptors calculated on their objects, and penalizing this dissimilarity in network training.
Experiments revealed that the proposed shape-aware loss function led to statistically significantly better results for liver segmentation, compared to its counterparts.
arXiv Detail & Related papers (2023-09-21T14:23:10Z) - Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution [91.3781512926942]
Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures.
This work investigates the potential of network pruning for super-resolution iteration to take advantage of off-the-shelf network designs and reduce the underlying computational overhead.
We propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly network at each and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly.
arXiv Detail & Related papers (2023-03-16T21:06:13Z) - A Systematic Performance Analysis of Deep Perceptual Loss Networks: Breaking Transfer Learning Conventions [5.470136744581653]
Deep perceptual loss is a type of loss function for images that computes the error between two images as the distance between deep features extracted from a neural network.
This work evaluates the effect of different pretrained loss networks on four different application areas.
arXiv Detail & Related papers (2023-02-08T13:08:51Z) - DTU-Net: Learning Topological Similarity for Curvilinear Structure
Segmentation [2.9398911304923447]
We present DTU-Net, a dual-decoder and topology-aware deep neural network consisting of two sequential light-weight U-Nets.
The texture net makes a coarse prediction using image texture information.
The topology net learns topological information from the coarse prediction by employing a triplet loss trained to recognize false and missed splits.
arXiv Detail & Related papers (2022-05-23T08:15:26Z) - Contrastive Learning for Label-Efficient Semantic Segmentation [44.10416030868873]
Convolutional Neural Network (CNN) based semantic segmentation approaches have achieved impressive results by using large amounts of labeled data.
Deep CNNs trained with the de facto cross-entropy loss can easily overfit to small amounts of labeled data.
We propose a simple and effective contrastive learning-based training strategy in which we first pretrain the network using a pixel-wise, label-based contrastive loss.
arXiv Detail & Related papers (2020-12-13T07:05:39Z) - Structured Convolutions for Efficient Neural Network Design [65.36569572213027]
We tackle model efficiency by exploiting redundancy in the textitimplicit structure of the building blocks of convolutional neural networks.
We show how this decomposition can be applied to 2D and 3D kernels as well as the fully-connected layers.
arXiv Detail & Related papers (2020-08-06T04:38:38Z) - On Robustness and Transferability of Convolutional Neural Networks [147.71743081671508]
Modern deep convolutional networks (CNNs) are often criticized for not generalizing under distributional shifts.
We study the interplay between out-of-distribution and transfer performance of modern image classification CNNs for the first time.
We find that increasing both the training set and model sizes significantly improve the distributional shift robustness.
arXiv Detail & Related papers (2020-07-16T18:39:04Z) - Perceptually Optimizing Deep Image Compression [53.705543593594285]
Mean squared error (MSE) and $ell_p$ norms have largely dominated the measurement of loss in neural networks.
We propose a different proxy approach to optimize image analysis networks against quantitative perceptual models.
arXiv Detail & Related papers (2020-07-03T14:33:28Z) - On the Predictability of Pruning Across Scales [29.94870276983399]
We show that the error of magnitude-pruned networks empirically follows a scaling law with interpretable coefficients that depend on the architecture and task.
As neural networks become ever larger and costlier to train, our findings suggest a framework for reasoning conceptually and analytically about a standard method for unstructured pruning.
arXiv Detail & Related papers (2020-06-18T15:41:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.