Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation

• URL: http://arxiv.org/abs/2608.24138v1
• Date: Tue, 25 Aug 2026 07:03:46 GMT
• Title: Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation
• Abstract summary: A local code edit propagates through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful.<n>We present RubSE, a rubric-guided framework that uses rubrics to represent visual feedback as a structured visual-repair context.<n>RubSE substantially outperforms nave self-evolution in final-round and best-round settings, achieving more stable refinement and a higher trajectory-level performance ceiling.
• Score: 147.1183324267404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large vision-language models have shown strong progress in UI-to-code generation, yet their test-time self-evolution remains unstable. We first identify a fundamental obstacle, termed visual repair coupling: a local code edit may propagate through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful. To address this issue, we present RubSE, a Rubric-guided Self-Evolution framework that uses rubrics to represent visual feedback as a structured visual-repair context. At each refinement round, RubSE generates typed candidate rubrics, selects one prioritized repair target, and stores previously selected rubrics as history, thereby steering each revision toward a well-scoped visual repair while discouraging repeated or over-broad changes. Evaluations across six VLMs and three UI-to-code benchmarks demonstrate that RubSE substantially outperforms naïve self-evolution in final-round and best-round settings, achieving more stable refinement trajectories and a higher trajectory-level performance ceiling. Further analysis shows that RubSE mitigates trajectory collapse by improving recovery from severe visual regressions, and that stronger rubric generators can transfer effective visual-repair guidance to weaker code improvers.

Related papers

• InsightSeg: Reusing Correction Insights for Guideline-Consistent Segmentation [4.501547677532766]
  We introduce InsightSeg, an episodic memory mechanism that converts successful correction episodes into reusable, visually grounded insights.<n>Across and Cityscapes, InsightSeg improves both first-pass and final guideline-consistent segmentation performance while requiring fewer refinement steps.
  arXiv  Detail & Related papers  (2026-09-02T02:22:35Z)
• MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification [26.741669879220684]
  MT-Web2Code is the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification.<n>We develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages.<n> Experiments on 13 coding agents reveal that current agents struggle to faithfully target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns.
  arXiv  Detail & Related papers  (2026-08-04T11:14:00Z)
• Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts [4.312910873423553]
  We study visual self-distillation for code-generated visual artifacts.<n>Visual-SDPO treats rendered visual feedback as privileged context for a weight-sharing teacher.<n>Visual-SDPO improves over the zero-shot base by more than 10 absolute points in the primary metric.
  arXiv  Detail & Related papers  (2026-06-09T02:28:39Z)
• LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition [51.70817823155725]
  We propose a novel landmark-guided contrastive learning network with vision-language enhancement for Facial Expression Recognition (FER)<n>LaCoVL-FER integrates geometric priors from facial landmarks and semantic priors from a vision-language model.<n>Experiments demonstrate that LaCoVL-FER outperforms state-of-the-art methods on three representative real-world FER datasets.
  arXiv  Detail & Related papers  (2026-05-19T13:15:41Z)
• IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework [20.964700751378547]
  Existing text-to-SVG generation methods do not incorporate visual perception of the final rendered image.<n>We propose an Introspective SVG Generation Framework (IntroSVG)<n>We show that our method achieves state-of-the-art performance across several key evaluation metrics.
  arXiv  Detail & Related papers  (2026-03-10T07:44:51Z)
• Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing [76.2602505940467]
  Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and hallucination.<n>Inspired by the human strategy of using a finger as a visual anchor'' to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR)<n>The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors.
  arXiv  Detail & Related papers  (2026-02-18T13:40:53Z)
• SVRepair: Structured Visual Reasoning for Automated Program Repair [17.545585659174773]
  Large language models (LLMs) have recently shown strong potential for Automated Program Repair (APR)<n>We propose textbfSVRepair, a multimodal APR framework with structured visual representation.<n> SVRepair first fine-tunes a vision-language model, textbfStructured Visual Representation (SVR), to uniformly transform heterogeneous visual artifacts into a emphsemantic scene graph.
  arXiv  Detail & Related papers  (2026-02-05T06:26:46Z)
• A Multimodal RAG Framework for Housing Damage Assessment: Collaborative Optimization of Image Encoding and Policy Vector Retrieval [0.0]
  After natural disasters, accurate evaluations of damage to housing are important for insurance claims response and planning of resources.<n>In this work, we introduce a novel multimodal retrieval-augmented generation (MM-RAG) framework.<n>The framework takes end-to-end training, and combines the comparison loss, the retrieval loss and the generation loss to form multi-task optimization objectives.
  arXiv  Detail & Related papers  (2025-09-10T01:58:07Z)
• End-to-End Vision Tokenizer Tuning [73.3065542220568]
  The vision tokenizer optimized for low-level reconstruction is to downstream tasks requiring varied representations and semantics.<n>The loss of the vision tokenization can be the representation bottleneck for target tasks.<n>We propose ETT, an end-to-end vision tokenizer tuning approach that enables joint optimization between vision tokenization and target autoregressive tasks.
  arXiv  Detail & Related papers  (2025-05-15T17:59:39Z)
• IQPFR: An Image Quality Prior for Blind Face Restoration and Beyond [56.99331967165238]
  Blind Face Restoration (BFR) addresses the challenge of reconstructing degraded low-quality (LQ) facial images into high-quality (HQ) outputs.<n>We propose a novel framework that incorporates an Image Quality Prior (IQP) derived from No-Reference Image Quality Assessment (NR-IQA) models.<n>Our method outperforms state-of-the-art techniques across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-03-12T11:39:51Z)
• CRACT: Cascaded Regression-Align-Classification for Robust Visual Tracking [97.84109669027225]
  We introduce an improved proposal refinement module, Cascaded Regression-Align- Classification (CRAC) CRAC yields new state-of-the-art performances on many benchmarks. In experiments on seven benchmarks including OTB-2015, UAV123, NfS, VOT-2018, TrackingNet, GOT-10k and LaSOT, our CRACT exhibits very promising results in comparison with state-of-the-art competitors.
  arXiv  Detail & Related papers  (2020-11-25T02:18:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.