ChArtist: Generating Pictorial Charts with Unified Spatial and Subject Control

• URL: http://arxiv.org/abs/2603.14209v1
• Date: Sun, 15 Mar 2026 03:55:44 GMT
• Title: ChArtist: Generating Pictorial Charts with Unified Spatial and Subject Control
• Abstract summary: A pictorial chart is an effective medium for visual storytelling, seamlessly integrating visual elements with data charts.<n>Current methods that extract dense structural cues from natural images are ill-suited as conditioning signals for pictorial chart generation.<n>We present ChArtist, a domain-specific diffusion model for generating pictorial charts automatically.
• Score: 9.055386884800525
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A pictorial chart is an effective medium for visual storytelling, seamlessly integrating visual elements with data charts. However, creating such images is challenging because the flexibility of visual elements often conflicts with the rigidity of chart structures. This process thus requires a creative deformation that maintains both data faithfulness and visual aesthetics. Current methods that extract dense structural cues from natural images (e.g., edge or depth maps) are ill-suited as conditioning signals for pictorial chart generation. We present ChArtist, a domain-specific diffusion model for generating pictorial charts automatically, offering two distinct types of control: 1) spatial control that aligns well with the chart structure, and 2) subject-driven control that respects the visual characteristics of a reference image. To achieve this, we introduce a skeleton-based spatial control representation. This representation encodes only the data-encoding information of the chart, allowing for the easy incorporation of reference visuals without a rigid outline constraint. We implement our method based on the Diffusion Transformer (DiT) and leverage an adaptive position encoding mechanism to manage these two controls. We further introduce Spatially Gated Attention to modulate the interaction between spatial control and subject control. To support the fine-tuning of pre-trained models for this task, we created a large-scale dataset of 30,000 triplets (skeleton, reference image, pictorial chart). We also propose a unified data accuracy metric to evaluate the data faithfulness of the generated charts. We believe this work demonstrates that current generative models can achieve data-driven visual storytelling by moving beyond general-purpose conditions to task-specific representations. Project page: https://chartist-ai.github.io/.

Related papers

• Semantic-Structural Alignment for Generative Pictorial Charts [59.45629259524998]
  We present a generative framework for the automated synthesis of pictorial charts.<n>We frame the problem as a dual-conditioned generation task guided by two parallel external control signals.<n>Our method produces pictorial charts that are both artistically compelling and structurally consistent.
  arXiv  Detail & Related papers  (2026-05-05T05:20:46Z)
• CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution [23.66187133157774]
  CharTide is a novel data-centric framework for chart-to-code generation.<n>It explicitly decoupling training into visual perception, pure-text code logic, and modality fusion streams.<n>Experiments on ChartMimic, Plot2Code, and ChartX show that CharTide-7B/8B significantly outperforms open-source baselines.
  arXiv  Detail & Related papers  (2026-04-24T03:39:51Z)
• Proportion and Perspective Control for Flow-Based Image Generation [1.9336815376402718]
  We introduce and evaluate two ControlNets specialized for artistic control.<n>A proportion ControlNet uses bounding boxes to dictate the position and scale of objects, and a perspective ControlNet employs vanishing lines to control the 3D geometry of the scene.
  arXiv  Detail & Related papers  (2025-10-13T13:34:28Z)
• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding.<n>Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• Text2Chart31: Instruction Tuning for Chart Generation with Automatic Feedback [37.275533538711436]
  We propose a hierarchical pipeline and a new dataset for chart generation.<n>Our dataset, Text2Chart31, includes 31 unique plot types referring to the Matplotlib library.<n>We introduce a reinforcement learning-based instruction tuning technique for chart generation tasks without requiring human feedback.
  arXiv  Detail & Related papers  (2024-10-05T07:25:56Z)
• Object-Attribute Binding in Text-to-Image Generation: Evaluation and Control [58.37323932401379]
  Current diffusion models create images given a text prompt as input but struggle to correctly bind attributes mentioned in the text to the right objects in the image. We propose focused cross-attention (FCA) that controls the visual attention maps by syntactic constraints found in the input sentence. We show substantial improvements in T2I generation and especially its attribute-object binding on several datasets.
  arXiv  Detail & Related papers  (2024-04-21T20:26:46Z)
• Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training [112.94542676251133]
  We propose to learn scene graph embeddings by directly optimizing their alignment with images. Specifically, we pre-train an encoder to extract both global and local information from scene graphs. The resulting method, called SGDiff, allows for the semantic manipulation of generated images by modifying scene graph nodes and connections.
  arXiv  Detail & Related papers  (2022-11-21T01:11:19Z)
• Graph-to-3D: End-to-End Generation and Manipulation of 3D Scenes Using Scene Graphs [85.54212143154986]
  Controllable scene synthesis consists of generating 3D information that satisfy underlying specifications. Scene graphs are representations of a scene composed of objects (nodes) and inter-object relationships (edges) We propose the first work that directly generates shapes from a scene graph in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-08-19T17:59:07Z)
• Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphs [74.88118535585903]
  We propose the Abstract Scene Graph structure to represent user intention in fine-grained level. From the ASG, we propose a novel ASG2Caption model, which is able to recognise user intentions and semantics in the graph. Our model achieves better controllability conditioning on ASGs than carefully designed baselines on both VisualGenome and MSCOCO datasets.
  arXiv  Detail & Related papers  (2020-03-01T03:34:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.