GALA: Multimodal Graph Alignment for Bug Localization in Automated Program Repair

• URL: http://arxiv.org/abs/2604.08089v1
• Date: Thu, 09 Apr 2026 11:06:25 GMT
• Title: GALA: Multimodal Graph Alignment for Bug Localization in Automated Program Repair
• Abstract summary: We propose GALA, a framework that shifts multimodal Automated Program Repair (APR) from implicit semantic guessing to explicit structural reasoning.<n>GALA operates in four stages: it first constructs an Image UI Graph to capture visual elements and their structural relationships; then performs file-level alignment by cross-referencing this UI graph with repository-level structures.<n>By enforcing both semantic and relational consistency across modalities, GALA establishes a highly accurate visual-to-code mapping.
• Score: 40.228155198574356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Model (LLM)-based Automated Program Repair (APR) has shown strong potential on textual benchmarks, yet struggles in multimodal scenarios where bugs are reported with GUI screenshots. Existing methods typically convert images into plain text, which discards critical spatial relationships and causes a severe disconnect between visual observations and code components, leading localization to degrade into imprecise keyword matching. To bridge this gap, we propose GALA (Graph Alignment for Localization in APR), a framework that shifts multimodal APR from implicit semantic guessing to explicit structural reasoning. GALA operates in four stages: it first constructs an Image UI Graph to capture visual elements and their structural relationships; then performs file-level alignment by cross-referencing this UI graph with repository-level structures (e.g., file references) to locate candidate files; next conducts function-level alignment by reasoning over fine-grained code dependencies (e.g., call graphs) to precisely ground visual elements to corresponding code components; and finally performs patch generation within the grounded code context based on the aligned files and functions. By systematically enforcing both semantic and relational consistency across modalities, GALA establishes a highly accurate visual-to-code mapping. Evaluations on the SWE-bench Multimodal benchmark demonstrate that GALA achieves state-of-the-art performance, highlighting the effectiveness of hierarchical structural alignment.

Related papers

• Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts [4.312910873423553]
  We study visual self-distillation for code-generated visual artifacts.<n>Visual-SDPO treats rendered visual feedback as privileged context for a weight-sharing teacher.<n>Visual-SDPO improves over the zero-shot base by more than 10 absolute points in the primary metric.
  arXiv  Detail & Related papers  (2026-06-09T02:28:39Z)
• Chain of Evidence: Pixel-Level Visual Attribution for Iterative Retrieval-Augmented Generation [19.889854990300595]
  Iterative Retrieval-Augmented Generation (iRAG) has emerged as a powerful paradigm for answering complex multi-hop questions.<n>We present textbfChain of Evidence (CoE), a retriever-agnostic visual attribution framework.
  arXiv  Detail & Related papers  (2026-05-02T06:40:42Z)
• MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning [37.02901476078596]
  We introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning.<n> MM-CondChain is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition.<n>A Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable.
  arXiv  Detail & Related papers  (2026-03-12T17:59:56Z)
• Visual Self-Refine: A Pixel-Guided Paradigm for Accurate Chart Parsing [76.2602505940467]
  Existing models often struggle with visually dense charts, leading to errors like data omission, misalignment, and hallucination.<n>Inspired by the human strategy of using a finger as a visual anchor'' to ensure accuracy when reading complex charts, we propose a new paradigm named Visual Self-Refine (VSR)<n>The core idea of VSR is to enable a model to generate pixel-level localization outputs, visualize them, and then feed these visualizations back to itself, allowing it to intuitively inspect and correct its own potential visual perception errors.
  arXiv  Detail & Related papers  (2026-02-18T13:40:53Z)
• SVRepair: Structured Visual Reasoning for Automated Program Repair [17.545585659174773]
  Large language models (LLMs) have recently shown strong potential for Automated Program Repair (APR)<n>We propose textbfSVRepair, a multimodal APR framework with structured visual representation.<n> SVRepair first fine-tunes a vision-language model, textbfStructured Visual Representation (SVR), to uniformly transform heterogeneous visual artifacts into a emphsemantic scene graph.
  arXiv  Detail & Related papers  (2026-02-05T06:26:46Z)
• Text2Graph VPR: A Text-to-Graph Expert System for Explainable Place Recognition in Changing Environments [0.0]
  Text2Graph VPR converts image sequences into textual scene descriptions.<n> Scene graphs capture objects, attributes and pairwise relations.<n>We demonstrate robust retrieval under severe appearance shifts.
  arXiv  Detail & Related papers  (2025-12-21T06:16:20Z)
• CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation [54.53371540755023]
  Reasoning segmentation seeks pixel-accurate masks for targets referenced by complex, often implicit instructions.<n>We present CORA, a semi-supervised reasoning segmentation framework that jointly learns from limited labeled data and a large corpus of unlabeled images.<n> CORA achieves state-of-the-art results, requiring as few as 100 labeled images on Cityscapes, a benchmark dataset for urban scene understanding.
  arXiv  Detail & Related papers  (2025-11-21T20:14:55Z)
• GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion [33.66085762717581]
  LLMs excel in localized code completion but struggle with repository-level tasks due to limited context windows.<n> GRACE builds a multi-level, multi-semantic code graph to capture both static and dynamic code semantics.<n>Experiments demonstrate that GRACE significantly outperforms state-of-the-art methods across all metrics.
  arXiv  Detail & Related papers  (2025-09-07T09:01:48Z)
• AlignCAT: Visual-Linguistic Alignment of Category and Attribute for Weakly Supervised Visual Grounding [56.972490764212175]
  Weakly supervised visual grounding aims to locate objects in images based on text descriptions.<n>Existing methods lack strong cross-modal reasoning to distinguish subtle semantic differences in text expressions.<n>We introduce AlignCAT, a novel query-based semantic matching framework for weakly supervised VG.
  arXiv  Detail & Related papers  (2025-08-05T08:16:35Z)
• RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning [63.599057862999]
  RefChartQA is a novel benchmark that integrates Chart Question Answering (ChartQA) with visual grounding.<n>Our experiments demonstrate that incorporating spatial awareness via grounding improves response accuracy by over 15%.
  arXiv  Detail & Related papers  (2025-03-29T15:50:08Z)
• Exploring Part-Informed Visual-Language Learning for Person Re-Identification [52.92511980835272]
  We propose Part-Informed Visual-language Learning ($pi$-VL) to enhance fine-grained visual features with part-informed language supervisions for ReID tasks.<n>$pi$-VL introduces a human parsing-guided prompt tuning strategy and a hierarchical visual-language alignment paradigm to ensure within-part feature semantic consistency.<n>As a plug-and-play and inference-free solution, our $pi$-VL achieves performance comparable to or better than state-of-the-art methods on four commonly used ReID benchmarks.
  arXiv  Detail & Related papers  (2023-08-04T23:13:49Z)
• SA-VQA: Structured Alignment of Visual and Semantic Representations for Visual Question Answering [29.96818189046649]
  We propose to apply structured alignments, which work with graph representation of visual and textual content. As demonstrated in our experimental results, such a structured alignment improves reasoning performance. The proposed model, without any pretraining, outperforms the state-of-the-art methods on GQA dataset, and beats the non-pretrained state-of-the-art methods on VQA-v2 dataset.
  arXiv  Detail & Related papers  (2022-01-25T22:26:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.