UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization

• URL: http://arxiv.org/abs/2606.31732v1
• Date: Tue, 30 Jun 2026 14:29:48 GMT
• Title: UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
• Abstract summary: We introduce UniCoder, a unified RL framework that integrates two novel mechanisms.<n>First, we propose textbfSymbolic Attribute Alignment, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes.<n>Second, to escape local optima, we devise textbfReference-Guided Code Optimization, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups.
• Score: 22.627858794105066
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise alignment that standard Multimodal Large Language Models (MLLMs) fail to achieve through Supervised Fine-Tuning (SFT) alone. While Reinforcement Learning (RL) offers a theoretical pathway to bridge this gap, its application is hindered by two fundamental obstacles: (1) \textit{Reward Coarseness}, where semantic metrics like CLIP scores fail to penalize fine-grained element deviations, and (2) \textit{Exploration Stagnation}, where the sparse, heterogeneous code search space prevents the policy from bootstrapping valid trajectories. To overcome these limitations, we introduce UniCoder, a unified RL framework that integrates two novel mechanisms. First, we propose \textbf{Symbolic Attribute Alignment}, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes (e.g., hex colors, coordinate limits), enabling dense, element-wise reward computation. Second, to escape local optima, we devise \textbf{Reference-Guided Code Optimization}, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups, transforming blind exploration into guided policy improvement. Extensive experiments on ChartMimic, UniSVG, Design2Code and ScreenBench benchmarks demonstrate that our 8B-parameter model not only surpasses all open-source baselines but also achieves state-of-the-art performance comparable to proprietary models, establishing a new paradigm for generalized visual-to-code synthesis.

Related papers

• Break the Optimization Barrier of LLM-Enhanced Recommenders: A Theoretical Analysis and Practical Framework [5.68373404216548]
  Training-Friendly LLM-Enhanced Recommender (TF-LLMER) is a lightweight framework with two key components.<n> Rec-PCA is a recommendation-aware dimensionality reduction method that injects collaborative structure into the representation transformation.<n> TF-LLMER significantly outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-04-22T12:22:47Z)
• CoDe-R: Refining Decompiler Output with LLMs via Rationale Guidance and Adaptive Inference [10.71961188176124]
  Binary decompilation is a critical reverse engineering task aimed at reconstructing high-level source code from stripped executables.<n>Large Language Models (LLMs) have recently shown promise, but they often suffer from "logical hallucinations" and "semantic misalignment"<n>We propose Cognitive Decompiler Refinement with Robustness (CoDe-R), a lightweight two-stage code refinement framework.
  arXiv  Detail & Related papers  (2026-04-14T15:58:38Z)
• Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought [55.65577137924979]
  We propose a framework that enables MLLMs to reason over images using continuous numerical coordinates.<n> NV-CoT expands the MLLM action space from discrete vocabulary tokens to a continuous Euclidean space.<n>Experiments on three benchmarks demonstrate that NV-CoT significantly improves localization precision and final answer accuracy.
  arXiv  Detail & Related papers  (2026-02-27T12:04:07Z)
• Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization [50.13408999553116]
  We propose RL-Text2Vis, the first reinforcement learning framework for Text2Vis generation.<n>Our method uses a novel multi-objective reward that jointly optimize textual accuracy, code validity, and visualization quality.<n>Our results establish GRPO as an effective strategy for structured, multimodal reasoning in visualization generation.
  arXiv  Detail & Related papers  (2026-01-08T04:29:07Z)
• VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning [13.193184888476404]
  We introduce textbfciCoder, a unified multimodal code generation model.<n>We begin by constructing a large-scale Supervised Finetuning (SFT) corpus comprising 1.6M image-code pairs.<n>We then introduce a Visual Reinforcement Learning (ViRL) strategy, which employs a coarse-to-fine reward mechanism to improve visual fidelity.
  arXiv  Detail & Related papers  (2025-11-01T04:05:26Z)
• IAR2: Improving Autoregressive Visual Generation with Semantic-Detail Associated Token Prediction [77.06211178777939]
  IAR2 is an advanced autoregressive framework that enables a hierarchical semantic-detail synthesis process.<n>We show that IAR2 sets a new state-of-the-art for autoregressive image generation, achieving a FID of 1.50 on ImageNet.
  arXiv  Detail & Related papers  (2025-10-08T12:08:21Z)
• Inpainting-Guided Policy Optimization for Diffusion Large Language Models [67.97530437998117]
  Masked diffusion large language models (dLLMs) are emerging as promising alternatives to autoregressive LLMs.<n>We explore how inpainting can inform RL algorithm design for dLLMs.
  arXiv  Detail & Related papers  (2025-09-12T16:44:31Z)
• Two Facets of the Same Optimization Coin: Model Degradation and Representation Collapse in Graph Foundation Models [41.39154912912367]
  GFMs are designed to learn the optimal embedding functions from multi-domain text-attributed graphs.<n> graph VQ-MAE stands out among the increasingly diverse landscape of GFM.<n>Despite its potential, domain generalization conflicts cause imperceptible pitfalls.
  arXiv  Detail & Related papers  (2025-09-10T08:45:08Z)
• Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinement [16.22363384653305]
  Multimodal Large Language Models (MLLMs) perform fine-grained visual parsing, precise code synthesis, and robust cross-modal reasoning.<n>We propose a dual preference-guided refinement framework that combines a feedback-driven, dual-modality reward mechanism with iterative preference learning.<n>Our framework significantly enhances the performance of general-purpose open-source MLLMs, enabling them to generate high-quality plotting code.
  arXiv  Detail & Related papers  (2025-04-03T07:51:20Z)
• Improving Autoregressive Visual Generation with Cluster-Oriented Token Prediction [52.09472099976885]
  IAR is an Improved AutoRegressive Visual Generation Method that enhances the training efficiency and generation quality of LLM-based visual generation models.<n>Our method consistently enhances the model training efficiency and performance from 100M to 1.4B, reducing the training time by half while achieving the same FID.
  arXiv  Detail & Related papers  (2025-01-01T15:58:51Z)
• GL-Fusion: Rethinking the Combination of Graph Neural Network and Large Language model [63.774726052837266]
  We introduce a new architecture that deeply integrates Graph Neural Networks (GNNs) with Large Language Models (LLMs)<n>We introduce three key innovations: (1) Structure-Aware Transformers, which incorporate GNN's message-passing capabilities directly into LLM's transformer layers; (2) Graph-Text Cross-Attention, which processes full, uncompressed text from graph nodes and edges; and (3) GNN-LLM Twin Predictor, enabling LLM's flexible autoregressive generation alongside GNN's scalable one-pass prediction.
  arXiv  Detail & Related papers  (2024-12-08T05:49:58Z)
• GLARE: Low Light Image Enhancement via Generative Latent Feature based Codebook Retrieval [80.96706764868898]
  We present a new Low-light Image Enhancement (LLIE) network via Generative LAtent feature based codebook REtrieval (GLARE) We develop a generative Invertible Latent Normalizing Flow (I-LNF) module to align the LL feature distribution to NL latent representations, guaranteeing the correct code retrieval in the codebook. Experiments confirm the superior performance of GLARE on various benchmark datasets and real-world data.
  arXiv  Detail & Related papers  (2024-07-17T09:40:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.