FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations

• URL: http://arxiv.org/abs/2411.10818v1
• Date: Sat, 16 Nov 2024 14:53:03 GMT
• Title: FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations
• Authors: Hmrishav Bandyopadhyay, Yi-Zhe Song,
• Abstract summary: Sketch animations offer a powerful medium for visual storytelling, from simple flip-book doodles to professional studio productions. We present FlipSketch, a system that brings back the magic of flip-book animation -- just draw your idea and describe how you want it to move!
• Score: 65.64014682930164
• License:
• Abstract: Sketch animations offer a powerful medium for visual storytelling, from simple flip-book doodles to professional studio productions. While traditional animation requires teams of skilled artists to draw key frames and in-between frames, existing automation attempts still demand significant artistic effort through precise motion paths or keyframe specification. We present FlipSketch, a system that brings back the magic of flip-book animation -- just draw your idea and describe how you want it to move! Our approach harnesses motion priors from text-to-video diffusion models, adapting them to generate sketch animations through three key innovations: (i) fine-tuning for sketch-style frame generation, (ii) a reference frame mechanism that preserves visual integrity of input sketch through noise refinement, and (iii) a dual-attention composition that enables fluid motion without losing visual consistency. Unlike constrained vector animations, our raster frames support dynamic sketch transformations, capturing the expressive freedom of traditional animation. The result is an intuitive system that makes sketch animation as simple as doodling and describing, while maintaining the artistic essence of hand-drawn animation.

Related papers

• AniClipart: Clipart Animation with Text-to-Video Priors [28.76809141136148]
  We introduce AniClipart, a system that transforms static images into high-quality motion sequences guided by text-to-video priors. Experimental results show that the proposed AniClipart consistently outperforms existing image-to-video generation models.
  arXiv  Detail & Related papers  (2024-04-18T17:24:28Z)
• Dynamic Typography: Bringing Text to Life via Video Diffusion Prior [73.72522617586593]
  We present an automated text animation scheme, termed "Dynamic Typography" It deforms letters to convey semantic meaning and infuses them with vibrant movements based on user prompts. Our technique harnesses vector graphics representations and an end-to-end optimization-based framework.
  arXiv  Detail & Related papers  (2024-04-17T17:59:55Z)
• Iterative Motion Editing with Natural Language [23.836693717924845]
  Text-to-motion diffusion models can generate realistic animations from text prompts, but do not support fine-grained motion editing controls. We present a method for using natural language to iteratively specify local edits to existing character animations. Our system can perform motion edits that respect the animator's editing intent, remain faithful to the original animation, and yield realistic character animation results.
  arXiv  Detail & Related papers  (2023-12-15T22:38:24Z)
• AnimateZero: Video Diffusion Models are Zero-Shot Image Animators [63.938509879469024]
  We propose AnimateZero to unveil the pre-trained text-to-video diffusion model, i.e., AnimateDiff. For appearance control, we borrow intermediate latents and their features from the text-to-image (T2I) generation. For temporal control, we replace the global temporal attention of the original T2V model with our proposed positional-corrected window attention.
  arXiv  Detail & Related papers  (2023-12-06T13:39:35Z)
• MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model [74.84435399451573]
  This paper studies the human image animation task, which aims to generate a video of a certain reference identity following a particular motion sequence. Existing animation works typically employ the frame-warping technique to animate the reference image towards the target motion. We introduce MagicAnimate, a diffusion-based framework that aims at enhancing temporal consistency, preserving reference image faithfully, and improving animation fidelity.
  arXiv  Detail & Related papers  (2023-11-27T18:32:31Z)
• Breathing Life Into Sketches Using Text-to-Video Priors [101.8236605955899]
  A sketch is one of the most intuitive and versatile tools humans use to convey their ideas visually. In this work, we present a method that automatically adds motion to a single-subject sketch. The output is a short animation provided in vector representation, which can be easily edited.
  arXiv  Detail & Related papers  (2023-11-21T18:09:30Z)
• AnimateAnything: Fine-Grained Open Domain Image Animation with Motion Guidance [13.416296247896042]
  We introduce an open domain image animation method that leverages the motion prior of video diffusion model. Our approach introduces targeted motion area guidance and motion strength guidance, enabling precise control of the movable area and its motion speed. We validate the effectiveness of our method through rigorous experiments on an open-domain dataset.
  arXiv  Detail & Related papers  (2023-11-21T03:47:54Z)
• SketchBetween: Video-to-Video Synthesis for Sprite Animation via Sketches [0.9645196221785693]
  2D animation is a common factor in game development, used for characters, effects and background art. Automated animation approaches exist, but are designed without animators in mind. We propose a problem formulation that adheres more closely to the standard workflow of animation.
  arXiv  Detail & Related papers  (2022-09-01T02:43:19Z)
• Deep Animation Video Interpolation in the Wild [115.24454577119432]
  In this work, we formally define and study the animation video code problem for the first time. We propose an effective framework, AnimeInterp, with two dedicated modules in a coarse-to-fine manner. Notably, AnimeInterp shows favorable perceptual quality and robustness for animation scenarios in the wild.
  arXiv  Detail & Related papers  (2021-04-06T13:26:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.