PQDAST: Depth-Aware Arbitrary Style Transfer for Games via Perceptual Quality-Guided Distillation

• URL: http://arxiv.org/abs/2502.16996v1
• Date: Mon, 24 Feb 2025 09:29:25 GMT
• Title: PQDAST: Depth-Aware Arbitrary Style Transfer for Games via Perceptual Quality-Guided Distillation
• Authors: Eleftherios Ioannou, Steve Maddock,
• Abstract summary: Artistic style transfer is concerned with the generation of imagery that combines the content of an image with the style of an artwork.<n>PQDAST is the first solution to integrate arbitrary style transfer into the game pipeline.
• Score: 0.6445605125467572
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Artistic style transfer is concerned with the generation of imagery that combines the content of an image with the style of an artwork. In the realm of computer games, most work has focused on post-processing video frames. Some recent work has integrated style transfer into the game pipeline, but it is limited to single styles. Integrating an arbitrary style transfer method into the game pipeline is challenging due to the memory and speed requirements of games. We present PQDAST, the first solution to address this. We use a perceptual quality-guided knowledge distillation framework and train a compressed model using the FLIP evaluator, which substantially reduces both memory usage and processing time with limited impact on stylisation quality. For better preservation of depth and fine details, we utilise a synthetic dataset with depth and temporal considerations during training. The developed model is injected into the rendering pipeline to further enforce temporal stability and avoid diminishing post-process effects. Quantitative and qualitative experiments demonstrate that our approach achieves superior performance in temporal consistency, with comparable style transfer quality, to state-of-the-art image, video and in-game methods.

Related papers

• Inversion-Free Video Style Transfer with Trajectory Reset Attention Control and Content-Style Bridging [5.501345898413532]
  We introduce Trajectory Reset Attention Control (TRAC), a novel method that allows for high-quality style transfer. TRAC operates by resetting the denoising trajectory and enforcing attention control, thus enhancing content consistency. We present a tuning-free framework that offers a stable, flexible, and efficient solution for both image and video style transfer.
  arXiv  Detail & Related papers  (2025-03-10T14:18:43Z)
• GIST: Towards Photorealistic Style Transfer via Multiscale Geometric Representations [9.514509577589449]
  GIST replaces the standard Neural Style Transfer autoencoding framework with a multiscale image expansion.<n>Our method matches multiresolution and multidirectional representations such as Wavelets and Contourlets by solving an optimal transport problem.
  arXiv  Detail & Related papers  (2024-12-03T07:05:39Z)
• UniVST: A Unified Framework for Training-free Localized Video Style Transfer [102.52552893495475]
  This paper presents UniVST, a unified framework for localized video style transfer based on diffusion model.<n>It operates without the need for training, offering a distinct advantage over existing diffusion methods that transfer style across entire videos.
  arXiv  Detail & Related papers  (2024-10-26T05:28:02Z)
• Blurry Video Compression: A Trade-off between Visual Enhancement and Data Compression [65.8148169700705]
  Existing video compression (VC) methods primarily aim to reduce the spatial and temporal redundancies between consecutive frames in a video. Previous works have achieved remarkable results on videos acquired under specific settings such as instant (known) exposure time and shutter speed. In this work, we tackle the VC problem in a general scenario where a given video can be blurry due to predefined camera settings or dynamics in the scene.
  arXiv  Detail & Related papers  (2023-11-08T02:17:54Z)
• Universal Photorealistic Style Transfer: A Lightweight and Adaptive Approach [18.46446576915766]
  Photorealistic style transfer aims to apply stylization while preserving the realism and structure of input content. We propose a Universal Photorealistic Style Transfer framework that delivers accurate photorealistic style transfer on high-resolution images and videos.
  arXiv  Detail & Related papers  (2023-09-18T07:04:40Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows [101.16791104543492]
  ArtFlow is proposed to prevent content leak during universal style transfer. It supports both forward and backward inferences and operates in a projection-transfer-reversion scheme. It achieves comparable performance to state-of-the-art style transfer methods while avoiding content leak.
  arXiv  Detail & Related papers  (2021-03-31T07:59:02Z)
• Learning Long-Term Style-Preserving Blind Video Temporal Consistency [6.6908747077585105]
  We propose a postprocessing model, to the transformation applied to videos, in the form of a recurrent neural network. Our model is trained using a Ping Pong procedure and its corresponding loss, recently introduced for GAN video generation. We evaluate our model on the DAVIS and videvo.net datasets and show that our approach offers state-of-the-art results concerning flicker removal.
  arXiv  Detail & Related papers  (2021-03-12T13:54:34Z)
• Arbitrary Video Style Transfer via Multi-Channel Correlation [84.75377967652753]
  We propose Multi-Channel Correction network (MCCNet) to fuse exemplar style features and input content features for efficient style transfer. MCCNet works directly on the feature space of style and content domain where it learns to rearrange and fuse style features based on similarity with content features. The outputs generated by MCC are features containing the desired style patterns which can further be decoded into images with vivid style textures.
  arXiv  Detail & Related papers  (2020-09-17T01:30:46Z)
• Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning [74.09149955786367]
  ArtNet can achieve universal, real-time, and high-quality style transfer on high-resolution images simultaneously. By using ArtNet and S2, our method is 2.3 to 107.4 times faster than state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-06-16T09:50:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.