Collaborative Distillation for Ultra-Resolution Universal Style Transfer

• URL: http://arxiv.org/abs/2003.08436v2
• Date: Tue, 24 Mar 2020 15:09:17 GMT
• Title: Collaborative Distillation for Ultra-Resolution Universal Style Transfer
• Authors: Huan Wang, Yijun Li, Yuehai Wang, Haoji Hu, Ming-Hsuan Yang
• Abstract summary: We present a new knowledge distillation method (named Collaborative Distillation) for encoder-decoder based neural style transfer. We achieve ultra-resolution (over 40 megapixels) universal style transfer on a 12GB GPU for the first time.
• Score: 71.18194557949634
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Universal style transfer methods typically leverage rich representations from deep Convolutional Neural Network (CNN) models (e.g., VGG-19) pre-trained on large collections of images. Despite the effectiveness, its application is heavily constrained by the large model size to handle ultra-resolution images given limited memory. In this work, we present a new knowledge distillation method (named Collaborative Distillation) for encoder-decoder based neural style transfer to reduce the convolutional filters. The main idea is underpinned by a finding that the encoder-decoder pairs construct an exclusive collaborative relationship, which is regarded as a new kind of knowledge for style transfer models. Moreover, to overcome the feature size mismatch when applying collaborative distillation, a linear embedding loss is introduced to drive the student network to learn a linear embedding of the teacher's features. Extensive experiments show the effectiveness of our method when applied to different universal style transfer approaches (WCT and AdaIN), even if the model size is reduced by 15.5 times. Especially, on WCT with the compressed models, we achieve ultra-resolution (over 40 megapixels) universal style transfer on a 12GB GPU for the first time. Further experiments on optimization-based stylization scheme show the generality of our algorithm on different stylization paradigms. Our code and trained models are available at https://github.com/mingsun-tse/collaborative-distillation.

Related papers

• Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models [66.1595537904019]
  Large language models (LLMs) can act as gradient priors in a zero-shot setting. We introduce LM-GC, a novel method that integrates LLMs with arithmetic coding.
  arXiv  Detail & Related papers  (2024-09-26T13:38:33Z)
• LinFusion: 1 GPU, 1 Minute, 16K Image [71.44735417472043]
  We introduce a low-rank approximation of a wide spectrum of popular linear token mixers. We find that the distilled model, termed LinFusion, achieves performance on par with or superior to the original SD. Experiments on SD-v1.5, SD-v2.1, and SD-XL demonstrate that LinFusion enables satisfactory and efficient zero-shot cross-resolution generation.
  arXiv  Detail & Related papers  (2024-09-03T17:54:39Z)
• Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution [25.558550480342614]
  This work presents MiDPK, a multi-granularity mixture of prior KDPK framework, to facilitate efficient image super-resolution model. Experiments demonstrate the effectiveness of the proposed MiDPK method.
  arXiv  Detail & Related papers  (2024-04-03T08:47:40Z)
• FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis [48.9652334528436]
  We introduce an innovative, training-free approach FouriScale from the perspective of frequency domain analysis. We replace the original convolutional layers in pre-trained diffusion models by incorporating a dilation technique along with a low-pass operation. Our method successfully balances the structural integrity and fidelity of generated images, achieving an astonishing capacity of arbitrary-size, high-resolution, and high-quality generation.
  arXiv  Detail & Related papers  (2024-03-19T17:59:33Z)
• Prompt-Based Exemplar Super-Compression and Regeneration for Class-Incremental Learning [22.676222987218555]
  Super-compression and regeneration method, ESCORT, substantially increases the quantity and enhances the diversity of exemplars. To minimize the domain gap between generated exemplars and real images, we propose partial compression and diffusion-based data augmentation.
  arXiv  Detail & Related papers  (2023-11-30T05:59:31Z)
• Strong Baseline and Bag of Tricks for COVID-19 Detection of CT Scans [2.696776905220987]
  Traditional deep learning frameworks encounter compatibility issues due to variations in slice numbers and resolutions in CT images. We propose a novel slice selection method for each CT dataset to address this limitation. In addition to the aforementioned methods, we explore various high-performance classification models, ultimately achieving promising results.
  arXiv  Detail & Related papers  (2023-03-15T09:52:28Z)
• Image-specific Convolutional Kernel Modulation for Single Image Super-resolution [85.09413241502209]
  In this issue, we propose a novel image-specific convolutional modulation kernel (IKM) We exploit the global contextual information of image or feature to generate an attention weight for adaptively modulating the convolutional kernels. Experiments on single image super-resolution show that the proposed methods achieve superior performances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-11-16T11:05:10Z)
• Learned Multi-Resolution Variable-Rate Image Compression with Octave-based Residual Blocks [15.308823742699039]
  We propose a new variable-rate image compression framework, which employs generalized octave convolutions (GoConv) and generalized octave transposed-convolutions (GoTConv) To enable a single model to operate with different bit rates and to learn multi-rate image features, a new objective function is introduced. Experimental results show that the proposed framework trained with variable-rate objective function outperforms the standard codecs such as H.265/HEVC-based BPG and state-of-the-art learning-based variable-rate methods.
  arXiv  Detail & Related papers  (2020-12-31T06:26:56Z)
• Model Fusion via Optimal Transport [64.13185244219353]
  We present a layer-wise model fusion algorithm for neural networks. We show that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.
  arXiv  Detail & Related papers  (2019-10-12T22:07:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.