PaintCopilot: Modeling Painting as Autonomous Artistic Continuation

• URL: http://arxiv.org/abs/2605.20941v1
• Date: Wed, 20 May 2026 09:27:06 GMT
• Title: PaintCopilot: Modeling Painting as Autonomous Artistic Continuation
• Abstract summary: PaintCopilot models painting as an open-ended autoregressive artistic behavior conditioned on evolving canvas states and prior brushstroke history.<n>PaintCopilot predicts future strokes directly from learned artistic dynamics.<n>We demonstrate that PaintCopilot enables fluid co-creative painting in which artists and AI continuously alternate control throughout the creative process.
• Score: 24.511616587268804
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present PaintCopilot, a co-creative neural painting assistant that models painting as an open-ended autoregressive artistic behavior conditioned on evolving canvas states and prior brushstroke history, without requiring a target image. Unlike existing neural painting methods that frame painting as pixel reconstruction toward a predefined reference, PaintCopilot predicts future strokes directly from learned artistic dynamics, analogous to how large language models continue text sequences from prior context. The framework proposes three complementary models: a ViT-based Target Predictor that infers artist intent from partial canvas observations, an autoregressive Next Stroke Predictor that generates temporally coherent brushstrokes via flow matching, and a VAE-based Region Sampler that synthesizes semantically localized stroke sequences on demand. Built on three differentiable brush representations (Hard Round, Brush Tip, and 2D Gaussian), the system supports four interactive workflows: Optimize History, Stroke Completion, Region Inpainting, and Dynamic Brush. Through case studies with professional artists, we demonstrate that PaintCopilot enables fluid co-creative painting workflows in which artists and AI continuously alternate control throughout the creative process.

Related papers

• VideoSketcher: Video Models Prior Enable Versatile Sequential Sketch Generation [73.23035143627598]
  Most generative models treat sketches as static images, overlooking the temporal structure that underlies creative drawing.<n>We present a data-efficient approach for sequential sketch generation that adapts pretrained text-to-video diffusion models.<n>Our method generates high-quality sketches that closely follow text-specified orderings while exhibiting rich visual detail.
  arXiv  Detail & Related papers  (2026-02-17T18:55:03Z)
• PaintFlow: A Unified Framework for Interactive Oil Paintings Editing and Generation [47.72342715926692]
  Oil painting is a high-level medium that blends human abstract thinking with artistic expression.<n>Existing generation and editing techniques are often constrained by the distribution of training data.<n>We introduce a unified multimodal framework for oil painting generation and editing.
  arXiv  Detail & Related papers  (2025-12-09T12:31:00Z)
• Loomis Painter: Reconstructing the Painting Process [56.713812157283805]
  Step-by-step painting tutorials are vital for learning artistic techniques, but existing video resources lack interactivity and personalization.<n>We propose a unified framework for multi-media painting process generation with a semantics-driven style control mechanism.<n>We also build a large-scale dataset of real painting processes and evaluate cross-media consistency, temporal coherence, and final-image fidelity.
  arXiv  Detail & Related papers  (2025-11-21T16:06:32Z)
• Birth of a Painting: Differentiable Brushstroke Reconstruction [25.61763988336406]
  Painting embodies a unique form of visual storytelling, where the creation process is as significant as the final artwork.<n>Our approach produces realistic and stylized appearances, offering a unified model for digital painting.
  arXiv  Detail & Related papers  (2025-11-17T09:55:53Z)
• Vectorized Region Based Brush Strokes for Artistic Rendering [3.5297361401370044]
  Recent stroke-based painting systems focus on capturing stroke details by predicting and iteratively refining stroke parameters.<n>These methods often struggle to produce stroke compositions that align with artistic principles and intent.<n>We propose an image-to-painting method that (i) facilitates semantic guidance for brush strokes in targeted regions, (ii) computes the brush stroke parameters, and (iii) establishes a sequence among segments and strokes to sequentially render the final painting.
  arXiv  Detail & Related papers  (2025-06-11T17:45:36Z)
• PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference [62.72779589895124]
  We make the first attempt to align diffusion models for image inpainting with human aesthetic standards via a reinforcement learning framework. We train a reward model with a dataset we construct, consisting of nearly 51,000 images annotated with human preferences. Experiments on inpainting comparison and downstream tasks, such as image extension and 3D reconstruction, demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-29T11:49:39Z)
• ProcessPainter: Learn Painting Process from Sequence Data [27.9875429986135]
  The painting process of artists is inherently stepwise and varies significantly among different painters and styles. Traditional stroke-based rendering methods break down images into sequences of brushstrokes, yet they fall short of replicating the authentic processes of artists. We introduce ProcessPainter, a text-to-video model that is initially pre-trained on synthetic data and subsequently fine-tuned with a select set of artists' painting sequences.
  arXiv  Detail & Related papers  (2024-06-10T07:18:41Z)
• Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region [66.75826549444909]
  Stroke-based rendering aims to recreate an image with a set of strokes. We propose Compositional Neural Painter, which predicts the painting region based on the current canvas. We extend our method to stroke-based style transfer with a novel differentiable distance transform loss.
  arXiv  Detail & Related papers  (2023-09-07T06:27:39Z)
• Interactive Neural Painting [66.9376011879115]
  This paper proposes the first approach for Interactive Neural Painting (NP) We propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder. Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art.
  arXiv  Detail & Related papers  (2023-07-31T07:02:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.