MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

• URL: http://arxiv.org/abs/2608.03474v1
• Date: Tue, 04 Aug 2026 11:14:00 GMT
• Title: MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification
• Abstract summary: MT-Web2Code is the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification.<n>We develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages.<n> Experiments on 13 coding agents reveal that current agents struggle to faithfully target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns.
• Score: 26.741669879220684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing benchmarks predominantly focus on single-turn full-page generation from scratch, overlooking the iterative workflow of real-world frontend engineering, where developers repeatedly reconstruct missing regions and modify localized elements within existing codebases. To bridge this gap, we introduce MT-Web2Code, the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification, which contains 102 tasks spanning 16 vertical domains. To construct deterministic repair trajectories without costly turn-level human annotation, we develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages. We further propose a dual-axis evaluation protocol that measures target-region fidelity and the preservation of unaffected content, where regional reconstruction is assessed by a 5-dimensional VLM-based rubric and localized modification by deterministic pixel-grounded alignment. Experiments on 13 frontier coding agents reveal that current agents struggle to faithfully reconstruct target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns. Beyond benchmarking, our deterministic evaluation metrics provide fine-grained feedback signals that may facilitate future research on training iterative UI coding agents. Our evaluation code and data will soon be released.

Related papers

• Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation [147.1183324267404]
  A local code edit propagates through layout, style, and component dependencies, correcting one visual mismatch while degrading regions that were previously faithful.<n>We present RubSE, a rubric-guided framework that uses rubrics to represent visual feedback as a structured visual-repair context.<n>RubSE substantially outperforms nave self-evolution in final-round and best-round settings, achieving more stable refinement and a higher trajectory-level performance ceiling.
  arXiv  Detail & Related papers  (2026-08-25T07:03:46Z)
• EgoHieraLoc: A Cortically Inspired Hierarchical Segmentation-Guided Framework for Egocentric Visual Query Localization [9.547789251222838]
  Visual query localization (VQL) aims to retrieve and re-localize a queried object in egocentric videos.<n>Human vision handles ambiguity through a hierarchical process.<n>We propose textbfEgoHieraLoc, a unified framework for VQL-2D and VQL-3D.
  arXiv  Detail & Related papers  (2026-08-10T14:31:01Z)
• PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents [62.36339598951133]
  We present PhoenixRepair, a multi-agent framework that explores multiple candidate edit locations and generates better patches.<n>PhoenixRepair achieves the largest relative improvement of 7.8% over SWE-agent under DeepSeek-V3.1, and attains the highest resolved rate of 76.0% Pass@1 under MiniMax-M2.5.
  arXiv  Detail & Related papers  (2026-07-21T08:49:30Z)
• SwinAD: Multi-stage feature reconstruction for unsupervised industrial anomaly detection [0.0]
  SwinAD is a reconstruction-based framework for multi-class unsupervised anomaly detection.<n> hierarchical encoder provides semantically rich multi-scale features.<n> Feature diversity-preserving reconstruction framework maintains complementary reconstruction hypotheses.
  arXiv  Detail & Related papers  (2026-07-16T03:39:56Z)
• Deep Unrolled Networks in Representation Space Applied to MRI Reconstruction [1.2806323642084227]
  Deeped networks (DUNs) integrate physical forward models with learned regularization in cascaded network architectures.<n>We introduce DUNE, a framework that maintains exact adherence to physical measurements while operating in learned representation spaces.
  arXiv  Detail & Related papers  (2026-06-19T17:01:41Z)
• Beyond Binary Success: A Diagnostic Meta-Evaluation Framework for Fine-Grained Manipulation [98.79811866787263]
  We introduce MetaFine, a diagnostic meta-evaluation framework that disentangles manipulation competency along three axes.<n>We identify the visual encoder's ability to preserve local spatial structure as a key bottleneck for fine-grained precision.<n>By shifting evaluation from ranking to diagnosis, MetaFine turns benchmarking into an actionable compass for repairing the layered capacities underlying genuine physical dexterity.
  arXiv  Detail & Related papers  (2026-05-19T15:25:13Z)
• Align then Refine: Text-Guided 3D Prostate Lesion Segmentation [5.284052813431228]
  Volumetric methods must combine multiple modalities while ensuring anatomical consistency.<n>Current models struggle to integrate cross-modal information reliably.<n>We propose a new multi-encoder U-Net architecture incorporating three key innovations.
  arXiv  Detail & Related papers  (2026-04-20T18:13:52Z)
• WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models [40.87133775066985]
  We introduce Web, a multimodal benchmark that provides unified lifecycle evaluation of web engineering capability.<n>Web spans three input modalities (text, image) and three task types (generation, editing, repair)<n>For evaluation, we adopt a checklist-guided LLM-as-a-Judge protocol for editing and repair, and propose a novel Agent-as-a-Judge paradigm for generation that autonomously generated websites in a real browser.
  arXiv  Detail & Related papers  (2026-04-20T13:09:38Z)
• Seeing the Unseen: Mask-Driven Positional Encoding and Strip-Convolution Context Modeling for Cross-View Object Geo-Localization [8.559240391514063]
  Cross-view object geo-localization enables high-precision object localization through cross-view matching.<n>Existing methods rely on keypoint-based positional encoding, which captures only 2D coordinates while neglecting object shape information.<n>We propose a mask-based positional encoding scheme that leverages segmentation masks to capture both spatial coordinates and object silhouettes.<n>We present EDGeo, an end-to-end framework for robust cross-view object geo-localization.
  arXiv  Detail & Related papers  (2025-10-23T06:07:07Z)
• Localizing Audio-Visual Deepfakes via Hierarchical Boundary Modeling [50.8215545241128]
  We propose a.<n> Boundary Modeling Network (HBMNet), which includes three modules: an Audio-Visual Feature, a.<n> Coarse Proposal Generator and a Fine-Hierarchical Probabilities Generator.<n>From the modality perspective, we enhance audio-visual encoding and fusion, reinforced by frame-level supervision.<n>Experiments show that encoding and fusion primarily improve precision, while frame-level supervision recall.
  arXiv  Detail & Related papers  (2025-08-04T02:41:09Z)
• Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization [52.87635234206178]
  This paper proposes a new framework, namely MoNFAP, specifically tailored for multi-face manipulation detection and localization. The framework incorporates two novel modules: the Forgery-aware Unified Predictor (FUP) Module and the Mixture-of-Noises Module (MNM)
  arXiv  Detail & Related papers  (2024-08-05T08:35:59Z)
• In-Domain GAN Inversion for Faithful Reconstruction and Editability [132.68255553099834]
  We propose in-domain GAN inversion, which consists of a domain-guided domain-regularized and a encoder to regularize the inverted code in the native latent space of the pre-trained GAN model. We make comprehensive analyses on the effects of the encoder structure, the starting inversion point, as well as the inversion parameter space, and observe the trade-off between the reconstruction quality and the editing property.
  arXiv  Detail & Related papers  (2023-09-25T08:42:06Z)
• DETR Doesn't Need Multi-Scale or Locality Design [69.56292005230185]
  This paper presents an improved DETR detector that maintains a "plain" nature. It uses a single-scale feature map and global cross-attention calculations without specific locality constraints. We show that two simple technologies are surprisingly effective within a plain design to compensate for the lack of multi-scale feature maps and locality constraints.
  arXiv  Detail & Related papers  (2023-08-03T17:59:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.