Merge-Bench: Resolve Merge Conflicts with Large Language Models

• URL: http://arxiv.org/abs/2605.25890v1
• Date: Mon, 25 May 2026 14:17:48 GMT
• Title: Merge-Bench: Resolve Merge Conflicts with Large Language Models
• Abstract summary: This paper applies machine learning to the difficult and important task of version control merging.<n>We constructed a dataset, Merge-Bench, of 7938 real-world merge conflict hunks from 1439 GitHub repositories.<n>We trained a model, LLMergeJ, to resolve merge conflicts in Java programs.
• Score: 1.8162282886873073
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper applies machine learning to the difficult and important task of version control merging. (1) We constructed a dataset, Merge-Bench, of 7938 real-world merge conflict hunks from 1439 GitHub repositories. The ground truth is the merge resolution that developers committed to the repository. Our dataset construction methodology is scalable to arbitrary amounts of data since no manual labeling is required. (2) We trained a model, LLMergeJ, to resolve merge conflicts in Java programs. Our approach uses Group Relative Policy Optimization (GRPO), an online reinforcement learning method, to train a Large Language Model (LLM). (3) We performed two evaluations of the performance of LLMs on resolving merge conflicts. On Java programs, LLMergeJ with 14B parameters outperforms 3 commercial LLMs, trailing only Gemini 2.5 Pro. Across 11 programming languages, commercial LLM performance is largely stable from language to language. The best models correctly resolve less than 60% of merge conflicts.

Related papers

• Can Large Language Models Resolve Real Java Merge Conflicts? An Evaluation with a Calibrated LLM-as-Judge [6.965857844586889]
  Large language models (LLMs) can produce a candidate resolution for almost any conflict.<n> measuring whether those resolutions are actually good at scale is hard.<n>We study both problems together on real Java merge conflicts from ConflictBench.
  arXiv  Detail & Related papers  (2026-07-30T04:38:17Z)
• LLM-based vs. Search-based Merge Conflict Resolution: An Empirical Study of Competing Paradigms [0.0]
  generative approaches based on Large Language Models (LLMs) and optimization approaches from Search-Based Software Engineering (SBSE)<n>This paper presents the first in-depth empirical study directly comparing these paradigms to identify their capabilities and limitations in real-world scenarios.
  arXiv  Detail & Related papers  (2026-05-15T21:34:32Z)
• SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution [56.9361004704428]
  Large Language Models (LLMs) have demonstrated remarkable proficiency across a variety of complex tasks.<n>SWE-Fixer is a novel open-source framework designed to effectively and efficiently resolve GitHub issues.<n>We assess our approach on the SWE-Bench Lite and Verified benchmarks, achieving competitive performance among open-source models.
  arXiv  Detail & Related papers  (2025-01-09T07:54:24Z)
• LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models [73.13933847198395]
  We propose a training-free framework for processing long texts, utilizing a divide-and-conquer strategy to achieve comprehensive document understanding. The proposed LLM$times$MapReduce framework splits the entire document into several chunks for LLMs to read and then aggregates the intermediate answers to produce the final output.
  arXiv  Detail & Related papers  (2024-10-12T03:13:44Z)
• CONGRA: Benchmarking Automatic Conflict Resolution [3.9910625211670485]
  ConGra is a benchmarking scheme designed to evaluate the performance of software merging tools. We build a large-scale evaluation dataset based on 44,948 conflicts from 34 real-world projects.
  arXiv  Detail & Related papers  (2024-09-21T12:21:41Z)
• Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement [72.97553348776425]
  We make a pioneering effort to broaden the applicability of merging techniques from FT to PT LLMs. We introduce an approach based on WeIght DisENtanglement (WIDEN) to effectively extend the merging scope. We merge Qwen1.5-Chat (an FT LLM with instruction-following skills) with Sailor (a PT LLM with multilingual abilities) across 7B and 14B model scales.
  arXiv  Detail & Related papers  (2024-08-06T10:46:46Z)
• WizardMerge -- Save Us From Merging Without Any Clues [8.21089093466603]
  We present WizardMerge, an auxiliary tool that leverages merging results from Git to retrieve code block dependency on text and LLVM-IR level. The outcomes demonstrate that WizardMerge diminishes conflict merging time costs, achieving a 23.85% reduction.
  arXiv  Detail & Related papers  (2024-07-03T05:40:29Z)
• Text-like Encoding of Collaborative Information in Large Language Models for Recommendation [58.87865271693269]
  We introduce BinLLM, a novel method to seamlessly integrate collaborative information with Large Language Models for Recommendation (LLMRec) BinLLM converts collaborative embeddings from external models into binary sequences. BinLLM provides options to compress the binary sequence using dot-decimal notation to avoid excessively long lengths.
  arXiv  Detail & Related papers  (2024-06-05T12:45:25Z)
• MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Time [51.5039731721706]
  MindStar is a purely inference-based searching method for large language models. It formulates reasoning tasks as searching problems and proposes two search ideas to identify the optimal reasoning paths. It significantly enhances the reasoning abilities of open-source models, such as Llama-2-13B and Mistral-7B, and achieves comparable performance to GPT-3.5 and Grok-1.
  arXiv  Detail & Related papers  (2024-05-25T15:07:33Z)
• LLM-Pruner: On the Structural Pruning of Large Language Models [65.02607075556742]
  Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. We tackle the compression of LLMs within the bound of two constraints: being task-agnostic and minimizing the reliance on the original training dataset. Our method, named LLM-Pruner, adopts structural pruning that selectively removes non-critical coupled structures.
  arXiv  Detail & Related papers  (2023-05-19T12:10:53Z)
• MergeBERT: Program Merge Conflict Resolution via Neural Transformers [11.460182185916704]
  Merge conflicts can stall pull requests and continuous integration pipelines for hours to several days. We introduce MergeBERT, a novel neural program merge framework based on the token-level three-way differencing and a transformer model. Our model achieves 64--69% precision of merge resolution synthesis, yielding nearly a 2x performance improvement over existing structured and neural program merge tools.
  arXiv  Detail & Related papers  (2021-08-31T21:37:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.