From Pen to Pixel: Translating Hand-Drawn Plots into Graphical APIs via a Novel Benchmark and Efficient Adapter

• URL: http://arxiv.org/abs/2603.26356v1
• Date: Fri, 27 Mar 2026 12:27:49 GMT
• Title: From Pen to Pixel: Translating Hand-Drawn Plots into Graphical APIs via a Novel Benchmark and Efficient Adapter
• Abstract summary: We introduce a hand-drawn plot dataset named HDpy-13 to improve the performance of graphical API recommendations for hand-drawn plot images.<n>We also propose Plot-Adapter that allows for the training and storage of separate adapters rather than requiring an entire model for each language and domain.
• Score: 2.98885096929089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As plots play a critical role in modern data visualization and analysis, Plot2API is launched to help non-experts and beginners create their desired plots by directly recommending graphical APIs from reference plot images by neural networks. However, previous works on Plot2API have primarily focused on the recommendation for standard plot images, while overlooking the hand-drawn plot images that are more accessible to non-experts and beginners. To make matters worse, both Plot2API models trained on standard plot images and powerful multi-modal large language models struggle to effectively recommend APIs for hand-drawn plot images due to the domain gap and lack of expertise. To facilitate non-experts and beginners, we introduce a hand-drawn plot dataset named HDpy-13 to improve the performance of graphical API recommendations for hand-drawn plot images. Additionally, to alleviate the considerable strain of parameter growth and computational resource costs arising from multi-domain and multi-language challenges in Plot2API, we propose Plot-Adapter that allows for the training and storage of separate adapters rather than requiring an entire model for each language and domain. In particular, Plot-Adapter incorporates a lightweight CNN block to improve the ability to capture local features and implements projection matrix sharing to reduce the number of fine-tuning parameters further. Experimental results demonstrate both the effectiveness of HDpy-13 and the efficiency of Plot-Adapter.

Related papers

• Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing [76.2602505940467]
  Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and hallucination.<n>Inspired by the human strategy of using a finger as a visual anchor'' to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR)<n>The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors.
  arXiv  Detail & Related papers  (2026-02-18T13:40:53Z)
• BigCharts-R1: Enhanced Chart Reasoning with Visual Reinforcement Finetuning [51.472854950300416]
  We propose BigCharts, a dataset creation pipeline that generates visually diverse chart images.<n>Unlike purely synthetic datasets, BigCharts incorporates real-world data, ensuring authenticity and visual diversity.<n>By introducing novel reward signals specifically designed for chart reasoning, our approach enhances model robustness and generalization.
  arXiv  Detail & Related papers  (2025-08-13T13:39:17Z)
• Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Images [56.134885746889026]
  semantic scene graph estimation methods utilize ground truth 3D annotations to accurately predict target objects, predicates, and relationships.<n>We overcome the noisy reconstructed pseudo point-based geometry from predicted depth maps and reduce the amount of background noise present in multi-view image features.<n>Our method outperforms current methods purely using multi-view images as the initial input.
  arXiv  Detail & Related papers  (2025-08-05T21:25:50Z)
• PointVDP: Learning View-Dependent Projection by Fireworks Rays for 3D Point Cloud Segmentation [66.00721801098574]
  We propose view-dependent projection (VDP) to facilitate point cloud segmentation.<n>VDP generates data-driven projections from 3D point distributions.<n>We construct color regularization to optimize the framework.
  arXiv  Detail & Related papers  (2025-07-09T07:44:00Z)
• TraGraph-GS: Trajectory Graph-based Gaussian Splatting for Arbitrary Large-Scale Scene Rendering [22.856356474506168]
  Existing methods typically partition large scenes into multiple regions, reconstruct a 3D representation using Gaussian splatting for each region, and eventually merge them for novel view rendering.<n>We propose TraGraph-GS, leveraging a trajectory graph to enable high-precision rendering for arbitrarily large-scale scenes.<n>Our method achieves an average improvement of 1.86 dB in PSNR on aerial datasets and 1.62 dB on ground datasets compared to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-06-10T11:43:04Z)
• T-Graph: Enhancing Sparse-view Camera Pose Estimation by Pairwise Translation Graph [3.3301244688278078]
  T-Graph is a lightweight, plug-and-play module to enhance camera pose estimation in sparse-view settings.<n>It takes paired image features as input and maps them through a Multilayer Perceptron (MLP)<n>It then constructs a fully connected translation graph, where nodes represent cameras and edges encode their translation relationships.
  arXiv  Detail & Related papers  (2025-05-02T11:50:48Z)
• GenPlot: Increasing the Scale and Diversity of Chart Derendering Data [0.0]
  We propose GenPlot, a plot generator that can generate billions of additional plots for chart-derendering using synthetic data. OCR-free chart-to-text translation has achieved state-of-the-art results on visual language tasks.
  arXiv  Detail & Related papers  (2023-06-20T17:25:53Z)
• Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Training [112.94542676251133]
  We propose to learn scene graph embeddings by directly optimizing their alignment with images. Specifically, we pre-train an encoder to extract both global and local information from scene graphs. The resulting method, called SGDiff, allows for the semantic manipulation of generated images by modifying scene graph nodes and connections.
  arXiv  Detail & Related papers  (2022-11-21T01:11:19Z)
• Plot2API: Recommending Graphic API from Plot via Semantic Parsing Guided Neural Network [13.936788648883068]
  Plot-based Graphic API recommendation (Plot2API) is an unstudied but meaningful issue. We present a novel deep multi-task learning approach named Semantic Parsing Guided Neural Network (SPGNN) In SPGNN, the recently advanced Convolutional Neural Network (CNN) named EfficientNet is employed as the backbone network for API recommendation.
  arXiv  Detail & Related papers  (2021-04-02T13:08:56Z)
• APEX-Net: Automatic Plot Extractor Network [24.299931323012757]
  We propose APEX-Net, a deep learning based framework with novel loss functions for solving the plot extraction problem. We introduce APEX-1M, a new large scale dataset which contains both the plot images and the raw data. We show visual results of our network on unseen plot images and demonstrate that it extracts the shape of the plots to a great extent.
  arXiv  Detail & Related papers  (2021-01-15T17:02:36Z)
• Graph-PCNN: Two Stage Human Pose Estimation with Graph Pose Refinement [54.29252286561449]
  We propose a two-stage graph-based and model-agnostic framework, called Graph-PCNN. In the first stage, heatmap regression network is applied to obtain a rough localization result, and a set of proposal keypoints, called guided points, are sampled. In the second stage, for each guided point, different visual feature is extracted by the localization. The relationship between guided points is explored by the graph pose refinement module to get more accurate localization results.
  arXiv  Detail & Related papers  (2020-07-21T04:59:15Z)
• A Grid-based Method for Removing Overlaps of Dimensionality Reduction Scatterplot Layouts [40.11095094521714]
  Distance Grid (DGrid) is a novel post-processing strategy to remove overlaps from Dimensionality Reduction (DR) scatterplot layouts. It faithfully preserves the original layout's characteristics and bounds the minimum glyph sizes.
  arXiv  Detail & Related papers  (2019-03-08T18:20:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.