Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation

• URL: http://arxiv.org/abs/2605.11307v1
• Date: Mon, 11 May 2026 22:50:12 GMT
• Title: Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation
• Abstract summary: We introduce Vision2Code, a reference-code-free benchmark and evaluation framework for image-to-code generation.<n>Vision2Code contains 2,169 test examples from 15 source datasets that span charts and plots, geometry, graphs, scientific imagery, documents, and 3D spatial scenes.<n>We report render-success diagnostics that separate code execution failures from reconstruction quality.
• Score: 18.596629746844567
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image-to-code generation tests whether a vision-language model (VLM) can recover the structure of an image enough to express it as executable code. Existing benchmarks either focus on narrow visual domains, depend on paired executable reference code, or rely on generic rubrics that miss domain-specific reconstruction errors. We introduce Vision2Code, a reference-code-free benchmark and evaluation framework for multi-domain image-to-code generation. Vision2Code contains 2,169 test examples from 15 source datasets that span charts and plots, geometry, graphs, scientific imagery, documents, and 3D spatial scenes. Models generate executable programs, which we render and score against the source image using a VLM rater with dataset-specific rubrics and deterministic guardrails for severe semantic failures. We report render-success diagnostics that separate code execution failures from reconstruction quality. Human validation shows that this evaluation protocol aligns better with human judgments than either a generic visual rubric or embedding-similarity baselines. Across nine open-weight and proprietary models, we find that image-to-code performance is domain-dependent: leading models perform well on regular chart- and graph-like visuals but remain weak on spatial scenes, chemistry, documents, and circuit-style diagrams. Finally, we show that evaluator-filtered model outputs can serve as training data to improve image-to-code capability, with Qwen3.5-9B improving from 1.60 to 1.86 on the benchmark without paired source programs. Vision2Code provides a reproducible testbed for measuring, diagnosing, and improving image-to-code generation. Our code and data are publicly available at https://image2code.github.io/vision2code/.

Related papers

• Visual-ERM: Reward Modeling for Visual Equivalence [59.317480168347664]
  Visual Equivalence Reward Model (Visual-ERM) is a multimodal generative reward model that provides fine-grained, interpretable, and task-agnostic feedback.<n>Visual-ERM improves Qwen3-VL-8B-Instruct by +8.4 on chart-to-code and yields consistent gains on table and SVG parsing.<n>VisualCritic-RewardBench (VC-RewardBench) is a benchmark for judging fine-grained image-to-image discrepancies on structured visual data.
  arXiv  Detail & Related papers  (2026-03-13T17:58:14Z)
• Multimodal OCR: Parse Anything from Documents [72.69545534962234]
  dots.mocr treats visual elements such as charts, diagrams, tables, and icons as first-class parsing targets.<n>It reconstructs both text and graphics as structured outputs, enabling more faithful document reconstruction.<n>It supports end-to-end training over heterogeneous document elements.
  arXiv  Detail & Related papers  (2026-03-13T14:42:21Z)
• Image Recognition with Vision and Language Embeddings of VLMs [14.022566577479322]
  Vision-language models (VLMs) have enabled strong zero-shot classification through image-text alignment.<n>We conduct a comprehensive evaluation of both language-guided and vision-only image classification with a diverse set of dual-encoder VLMs.<n>Key factors affecting accuracy are analysed, including prompt design, class diversity, the number of neighbours in k-NN, and reference set size.
  arXiv  Detail & Related papers  (2025-09-11T09:54:25Z)
• VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation [69.35779796364413]
  We present VisCode-200K, a large-scale instruction tuning dataset for Python-based visualization and self-correction.<n>It contains over 200K examples from two sources: (1) validated plotting code from open-source repositories, paired with natural language instructions and rendered plots; and (2) 45K multi-turn correction dialogues from Code-Feedback.
  arXiv  Detail & Related papers  (2025-06-04T13:24:44Z)
• TokBench: Evaluating Your Visual Tokenizer before Visual Generation [75.38270351179018]
  We analyze text and face reconstruction quality across various scales for different image tokenizers and VAEs.<n>Our results show modern visual tokenizers still struggle to preserve fine-grained features, especially at smaller scales.
  arXiv  Detail & Related papers  (2025-05-23T17:52:16Z)
• Image2Struct: Benchmarking Structure Extraction for Vision-Language Models [57.531922659664296]
  Image2Struct is a benchmark to evaluate vision-pixel models (VLMs) on extracting structure from images. In Image2Struct, VLMs are prompted to generate the underlying structure from an input image. The structure is then rendered to produce an output image, which is compared against the input image to produce a similarity score.
  arXiv  Detail & Related papers  (2024-10-29T18:44:59Z)
• Transformer-based Image Generation from Scene Graphs [11.443097632746763]
  Graph-structured scene descriptions can be efficiently used in generative models to control the composition of the generated image. Previous approaches are based on the combination of graph convolutional networks and adversarial methods for layout prediction and image generation. We show how employing multi-head attention to encode the graph information can improve the quality of the sampled data.
  arXiv  Detail & Related papers  (2023-03-08T14:54:51Z)
• Re-Imagen: Retrieval-Augmented Text-to-Image Generator [58.60472701831404]
  Retrieval-Augmented Text-to-Image Generator (Re-Imagen) Retrieval-Augmented Text-to-Image Generator (Re-Imagen)
  arXiv  Detail & Related papers  (2022-09-29T00:57:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.