Animating Through Warping: an Efficient Method for High-Quality Facial Expression Animation

• URL: http://arxiv.org/abs/2008.00362v1
• Date: Sat, 1 Aug 2020 23:52:33 GMT
• Title: Animating Through Warping: an Efficient Method for High-Quality Facial Expression Animation
• Abstract summary: We propose a novel framework known as Animating Through Warping (ATW) to enable efficient animation of HD images. It only requires the generator to be trained on small images and can do inference on an image of any size. Experiments show the effectiveness and efficiency of our method in generating high-resolution animations.
• Score: 11.522437263308984
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Advances in deep neural networks have considerably improved the art of animating a still image without operating in 3D domain. Whereas, prior arts can only animate small images (typically no larger than 512x512) due to memory limitations, difficulty of training and lack of high-resolution (HD) training datasets, which significantly reduce their potential for applications in movie production and interactive systems. Motivated by the idea that HD images can be generated by adding high-frequency residuals to low-resolution results produced by a neural network, we propose a novel framework known as Animating Through Warping (ATW) to enable efficient animation of HD images. Specifically, the proposed framework consists of two modules, a novel two-stage neural-network generator and a novel post-processing module known as Animating Through Warping (ATW). It only requires the generator to be trained on small images and can do inference on an image of any size. During inference, an HD input image is decomposed into a low-resolution component(128x128) and its corresponding high-frequency residuals. The generator predicts the low-resolution result as well as the motion field that warps the input face to the desired status (e.g., expressions categories or action units). Finally, the ResWarp module warps the residuals based on the motion field and adding the warped residuals to generates the final HD results from the naively up-sampled low-resolution results. Experiments show the effectiveness and efficiency of our method in generating high-resolution animations. Our proposed framework successfully animates a 4K facial image, which has never been achieved by prior neural models. In addition, our method generally guarantee the temporal coherency of the generated animations. Source codes will be made publicly available.

Related papers

• Instant Expressive Gaussian Head Avatar via 3D-Aware Expression Distillation [46.27695095774081]
  2D diffusion-based methods often compromise 3D consistency and speed.<n>3D-aware facial animation feedforward methods ensure 3D consistency and achieve faster inference speed.<n>Our method runs at 107.31 FPS for animation and pose control while achieving comparable animation quality to the state-of-the-art.
  arXiv  Detail & Related papers  (2025-12-18T18:53:28Z)
• LINR Bridge: Vector Graphic Animation via Neural Implicits and Video Diffusion Priors [8.74744557581647]
  We propose a novel method that integrates implicit neural representations with text-to-video diffusion models for vector graphic animation.<n>Our approach employs layered implicit neural representations to reconstruct vector graphics, preserving their inherent properties such as infinite resolution and precise color and shape constraints.<n>The neural representations are then optimized using video score distillation sampling, which leverages motion priors from pretrained text-to-video diffusion models.
  arXiv  Detail & Related papers  (2025-09-09T08:04:36Z)
• UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation [53.16986875759286]
  We present a UniAnimate framework to enable efficient and long-term human video generation. We map the reference image along with the posture guidance and noise video into a common feature space. We also propose a unified noise input that supports random noised input as well as first frame conditioned input.
  arXiv  Detail & Related papers  (2024-06-03T10:51:10Z)
• Generalizable Neural Human Renderer [29.533330467339013]
  Generalizable Human Renderer (GNH) is a novel method for rendering animatable humans from monocular video without any test-time optimization. We demonstrate GNH surpasses current state-of-the-art methods, notably achieving a 31.3% improvement in LPIPS.
  arXiv  Detail & Related papers  (2024-04-22T14:09:53Z)
• FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces [21.946327323788275]
  3D rendering of dynamic face is a challenging problem. We present a novel representation that enables high-quality rendering of an actor's dynamic facial performances.
  arXiv  Detail & Related papers  (2024-04-22T00:44:13Z)
• AnimateMe: 4D Facial Expressions via Diffusion Models [72.63383191654357]
  Recent advances in diffusion models have enhanced the capabilities of generative models in 2D animation. We employ Graph Neural Networks (GNNs) as denoising diffusion models in a novel approach, formulating the diffusion process directly on the mesh space. This facilitates the generation of facial deformations through a mesh-diffusion-based model.
  arXiv  Detail & Related papers  (2024-03-25T21:40:44Z)
• Hybrid Neural Rendering for Large-Scale Scenes with Motion Blur [68.24599239479326]
  We develop a hybrid neural rendering model that makes image-based representation and neural 3D representation join forces to render high-quality, view-consistent images. Our model surpasses state-of-the-art point-based methods for novel view synthesis.
  arXiv  Detail & Related papers  (2023-04-25T08:36:33Z)
• HQ3DAvatar: High Quality Controllable 3D Head Avatar [65.70885416855782]
  This paper presents a novel approach to building highly photorealistic digital head avatars. Our method learns a canonical space via an implicit function parameterized by a neural network. At test time, our method is driven by a monocular RGB video.
  arXiv  Detail & Related papers  (2023-03-25T13:56:33Z)
• Dynamic Neural Portraits [58.480811535222834]
  We present Dynamic Neural Portraits, a novel approach to the problem of full-head reenactment. Our method generates photo-realistic video portraits by explicitly controlling head pose, facial expressions and eye gaze. Our experiments demonstrate that the proposed method is 270 times faster than recent NeRF-based reenactment methods.
  arXiv  Detail & Related papers  (2022-11-25T10:06:14Z)
• Human Performance Modeling and Rendering via Neural Animated Mesh [40.25449482006199]
  We bridge the traditional mesh with a new class of neural rendering. In this paper, we present a novel approach for rendering human views from video. We demonstrate our approach on various platforms, inserting virtual human performances into AR headsets.
  arXiv  Detail & Related papers  (2022-09-18T03:58:00Z)
• Thin-Plate Spline Motion Model for Image Animation [9.591298403129532]
  Image animation brings life to the static object in the source image according to the driving video. Recent works attempt to perform motion transfer on arbitrary objects through unsupervised methods without using a priori knowledge. It remains a significant challenge for current unsupervised methods when there is a large pose gap between the objects in the source and driving images.
  arXiv  Detail & Related papers  (2022-03-27T18:40:55Z)
• Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation [99.64565200170897]
  We propose a novel human video synthesis method by explicitly disentangling the learning of time-coherent fine-scale details from the embedding of the human in 2D screen space. We show several applications of our approach, such as human reenactment and novel view synthesis from monocular video, where we show significant improvement over the state of the art both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2020-01-14T18:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.