Look, Compare and Draw: Differential Query Transformer for Automatic Oil Painting

• URL: http://arxiv.org/abs/2603.27720v1
• Date: Sun, 29 Mar 2026 14:52:10 GMT
• Title: Look, Compare and Draw: Differential Query Transformer for Automatic Oil Painting
• Abstract summary: This work introduces a new approach to automatic oil painting that emphasizes the creation of dynamic and expressive brushstrokes.<n>We incorporate differential image analysis into a neural oil painting model, allowing the model to concentrate on the incremental impact of successive brushstrokes.
• Score: 40.70919374645325
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work introduces a new approach to automatic oil painting that emphasizes the creation of dynamic and expressive brushstrokes. A pivotal challenge lies in mitigating the duplicate and common-place strokes, which often lead to less aesthetic outcomes. Inspired by the human painting process, \ie, observing, comparing, and drawing, we incorporate differential image analysis into a neural oil painting model, allowing the model to effectively concentrate on the incremental impact of successive brushstrokes. To operationalize this concept, we propose the Differential Query Transformer (DQ-Transformer), a new architecture that leverages differentially derived image representations enriched with positional encoding to guide the stroke prediction process. This integration enables the model to maintain heightened sensitivity to local details, resulting in more refined and nuanced stroke generation. Furthermore, we incorporate adversarial training into our framework, enhancing the accuracy of stroke prediction and thereby improving the overall realism and fidelity of the synthesized paintings. Extensive qualitative evaluations, complemented by a controlled user study, validate that our DQ-Transformer surpasses existing methods in both visual realism and artistic authenticity, typically achieving these results with fewer strokes. The stroke-by-stroke painting animations are available on our project website.

Related papers

• Enhancing Authorship Attribution with Synthetic Paintings [0.0]
  We propose a hybrid approach that combines real and synthetic data to enhance model accuracy and generalization across artistic styles.<n> Experimental results show that adding synthetic images leads to higher ROC-AUC and accuracy compared to using only real paintings.
  arXiv  Detail & Related papers  (2026-03-04T18:00:42Z)
• PaintFlow: A Unified Framework for Interactive Oil Paintings Editing and Generation [47.72342715926692]
  Oil painting is a high-level medium that blends human abstract thinking with artistic expression.<n>Existing generation and editing techniques are often constrained by the distribution of training data.<n>We introduce a unified multimodal framework for oil painting generation and editing.
  arXiv  Detail & Related papers  (2025-12-09T12:31:00Z)
• Vectorized Region Based Brush Strokes for Artistic Rendering [3.5297361401370044]
  Recent stroke-based painting systems focus on capturing stroke details by predicting and iteratively refining stroke parameters.<n>These methods often struggle to produce stroke compositions that align with artistic principles and intent.<n>We propose an image-to-painting method that (i) facilitates semantic guidance for brush strokes in targeted regions, (ii) computes the brush stroke parameters, and (iii) establishes a sequence among segments and strokes to sequentially render the final painting.
  arXiv  Detail & Related papers  (2025-06-11T17:45:36Z)
• Emergence of Painting Ability via Recognition-Driven Evolution [49.666177849272856]
  We present a model with a stroke branch and a palette branch that together simulate human-like painting.<n>We quantify the efficiency of visual communication by measuring the recognition accuracy achieved with machine vision.<n> Experimental results show that our model achieves superior performance in high-level recognition tasks.
  arXiv  Detail & Related papers  (2025-01-09T04:37:31Z)
• PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preference [62.72779589895124]
  We make the first attempt to align diffusion models for image inpainting with human aesthetic standards via a reinforcement learning framework. We train a reward model with a dataset we construct, consisting of nearly 51,000 images annotated with human preferences. Experiments on inpainting comparison and downstream tasks, such as image extension and 3D reconstruction, demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-10-29T11:49:39Z)
• AttentionPainter: An Efficient and Adaptive Stroke Predictor for Scene Painting [82.54770866332456]
  Stroke-based Rendering (SBR) aims to decompose an input image into a sequence of parameterized strokes, which can be rendered into a painting that resembles the input image. We propose AttentionPainter, an efficient and adaptive model for single-step neural painting.
  arXiv  Detail & Related papers  (2024-10-21T18:36:45Z)
• BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion [61.90969199199739]
  BrushNet is a novel plug-and-play dual-branch model engineered to embed pixel-level masked image features into any pre-trained DM. BrushNet's superior performance over existing models across seven key metrics, including image quality, mask region preservation, and textual coherence.
  arXiv  Detail & Related papers  (2024-03-11T17:59:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.