SketchPlay: Intuitive Creation of Physically Realistic VR Content with Gesture-Driven Sketching

• URL: http://arxiv.org/abs/2512.22016v1
• Date: Fri, 26 Dec 2025 12:32:39 GMT
• Title: SketchPlay: Intuitive Creation of Physically Realistic VR Content with Gesture-Driven Sketching
• Abstract summary: SketchPlay is a novel VR interaction framework that transforms humans' air-drawn sketches and gestures into dynamic, physically realistic scenes.<n>By combining these complementary forms of input, SketchPlay captures both the structure and dynamics of user-created content.
• Score: 12.374043359550775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creating physically realistic content in VR often requires complex modeling tools or predefined 3D models, textures, and animations, which present significant barriers for non-expert users. In this paper, we propose SketchPlay, a novel VR interaction framework that transforms humans' air-drawn sketches and gestures into dynamic, physically realistic scenes, making content creation intuitive and playful like drawing. Specifically, sketches capture the structure and spatial arrangement of objects and scenes, while gestures convey physical cues such as velocity, direction, and force that define movement and behavior. By combining these complementary forms of input, SketchPlay captures both the structure and dynamics of user-created content, enabling the generation of a wide range of complex physical phenomena, such as rigid body motion, elastic deformation, and cloth dynamics. Experimental results demonstrate that, compared to traditional text-driven methods, SketchPlay offers significant advantages in expressiveness, and user experience. By providing an intuitive and engaging creation process, SketchPlay lowers the entry barrier for non-expert users and shows strong potential for applications in education, art, and immersive storytelling.

Related papers

• 4-Doodle: Text to 3D Sketches that Move! [60.89021458068987]
  4-Doodle is the first training-free framework for generating dynamic 3D sketches from text.<n>Our method produces temporally realistic and structurally stable 3D sketch animations, outperforming existing baselines in both fidelity and controllability.
  arXiv  Detail & Related papers  (2025-10-29T09:33:29Z)
• X-Dyna: Expressive Dynamic Human Image Animation [49.896933584815926]
  X-Dyna is a zero-shot, diffusion-based pipeline for animating a single human image.<n>It generates realistic, context-aware dynamics for both the subject and the surrounding environment.
  arXiv  Detail & Related papers  (2025-01-17T08:10:53Z)
• FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations [65.64014682930164]
  Sketch animations offer a powerful medium for visual storytelling, from simple flip-book doodles to professional studio productions. We present FlipSketch, a system that brings back the magic of flip-book animation -- just draw your idea and describe how you want it to move!
  arXiv  Detail & Related papers  (2024-11-16T14:53:03Z)
• Dynamic Typography: Bringing Text to Life via Video Diffusion Prior [73.72522617586593]
  We present an automated text animation scheme, termed "Dynamic Typography" It deforms letters to convey semantic meaning and infuses them with vibrant movements based on user prompts. Our technique harnesses vector graphics representations and an end-to-end optimization-based framework.
  arXiv  Detail & Related papers  (2024-04-17T17:59:55Z)
• VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality [39.53150683721031]
  Our proposed VR-GS system represents a leap forward in human-centered 3D content interaction. The components of our Virtual Reality system are designed for high efficiency and effectiveness.
  arXiv  Detail & Related papers  (2024-01-30T01:28:36Z)
• Learning-based pose edition for efficient and interactive design [55.41644538483948]
  In computer-aided animation artists define the key poses of a character by manipulating its skeletons. Character pose must respect many ill-defined constraints, and so the resulting realism greatly depends on the animator's skill and knowledge. We describe an efficient tool for pose design, allowing users to intuitively manipulate a pose to create character animations.
  arXiv  Detail & Related papers  (2021-07-01T12:15:02Z)
• Real-time Deep Dynamic Characters [95.5592405831368]
  We propose a deep videorealistic 3D human character model displaying highly realistic shape, motion, and dynamic appearance. We use a novel graph convolutional network architecture to enable motion-dependent deformation learning of body and clothing. We show that our model creates motion-dependent surface deformations, physically plausible dynamic clothing deformations, as well as video-realistic surface textures at a much higher level of detail than previous state of the art approaches.
  arXiv  Detail & Related papers  (2021-05-04T23:28:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.