Back to the Basics: Rethinking Issue-Commit Linking with LLM-Assisted Retrieval

• URL: http://arxiv.org/abs/2507.09199v1
• Date: Sat, 12 Jul 2025 08:42:10 GMT
• Title: Back to the Basics: Rethinking Issue-Commit Linking with LLM-Assisted Retrieval
• Authors: Huihui Huang, Ratnadira Widyasari, Ting Zhang, Ivana Clairine Irsan, Jieke Shi, Han Wei Ang, Frank Liauw, Eng Lieh Ouh, Lwin Khin Shar, Hong Jin Kang, David Lo,
• Abstract summary: Issue-commit linking, which connects issues with commits that fix them, is crucial for software maintenance.<n>We propose EasyLink, which utilizes a vector database as a modern Information Retrieval technique.<n>Under our evaluation, EasyLink achieves an average Precision@1 of 75.91%, improving over the state-of-the-art by over four times.
• Score: 12.213080309713574
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Issue-commit linking, which connects issues with commits that fix them, is crucial for software maintenance. Existing approaches have shown promise in automatically recovering these links. Evaluations of these techniques assess their ability to identify genuine links from plausible but false links. However, these evaluations overlook the fact that, in reality, when a repository has more commits, the presence of more plausible yet unrelated commits may interfere with the tool in differentiating the correct fix commits. To address this, we propose the Realistic Distribution Setting (RDS) and use it to construct a more realistic evaluation dataset that includes 20 open-source projects. By evaluating tools on this dataset, we observe that the performance of the state-of-the-art deep learning-based approach drops by more than half, while the traditional Information Retrieval method, VSM, outperforms it. Inspired by these observations, we propose EasyLink, which utilizes a vector database as a modern Information Retrieval technique. To address the long-standing problem of the semantic gap between issues and commits, EasyLink leverages a large language model to rerank the commits retrieved from the database. Under our evaluation, EasyLink achieves an average Precision@1 of 75.91%, improving over the state-of-the-art by over four times. Additionally, this paper provides practical guidelines for advancing research in issue-commit link recovery.

Related papers

• DiffuRank: Effective Document Reranking with Diffusion Language Models [71.16830004674513]
  We propose DiffuRank, a reranking framework built upon diffusion language models (dLLMs)<n>dLLMs support more flexible decoding and generation processes that are not constrained to a left-to-right order.<n>We show dLLMs achieve performance comparable to, and in some cases exceeding, that of autoregressive LLMs with similar model sizes.
  arXiv  Detail & Related papers  (2026-02-13T02:18:14Z)
• Orion-RAG: Path-Aligned Hybrid Retrieval for Graphless Data [13.307131500057862]
  Retrieval-Augmented Generation (RAG) has proven effective for knowledge synthesis, yet it encounters significant challenges in practical scenarios where data is inherently discrete and fragmented.<n>We present Orion-RAG, which transforms fragmented documents into semi-structured data, enabling the system to link information across different files effectively.
  arXiv  Detail & Related papers  (2026-01-08T09:32:01Z)
• Establishing Traceability Links between Release Notes & Software Artifacts: Practitioners' Perspectives [5.70062525101025]
  In open-source environments where contributors work remotely and asynchronously, establishing and maintaining traceability links is often error-prone.<n>Our empirical study of GitHub repositories revealed that 47% of release artifacts lacked traceability links, and 12% contained broken links.<n>We implemented LLM-based approaches to automatically establish traceability links of three pairs between release note contents & PRs, release note contents & PRs and release note contents & issues.
  arXiv  Detail & Related papers  (2025-11-22T20:45:24Z)
• ABCD-LINK: Annotation Bootstrapping for Cross-Document Fine-Grained Links [57.514511353084565]
  We introduce a new domain-agnostic framework for selecting a best-performing approach and annotating cross-document links.<n>We apply our framework in two distinct domains -- peer review and news.<n>The resulting novel datasets lay foundation for numerous cross-document tasks like media framing and peer review.
  arXiv  Detail & Related papers  (2025-09-01T11:32:24Z)
• LinkAnchor: An Autonomous LLM-Based Agent for Issue-to-Commit Link Recovery [1.5399429731150376]
  A study on GitHub shows that only 42.2% of the issues are correctly linked to their commits.<n>We present LinkAnchor, the first autonomous LLM-based agent designed for issue-to-commit link recovery.
  arXiv  Detail & Related papers  (2025-08-17T04:21:44Z)
• The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason [1.6249398255272318]
  We introduce two diagnostic tasks: file path identification from issue descriptions alone, and ground truth function reproduction with only the current file context and issue description.<n>We show that state-of-the-art models achieve up to 76% accuracy in identifying buggy file paths using only issue descriptions, without access to repository structure.<n>This performance is merely up to 53% on tasks from repositories not included in SWE-Bench, pointing to possible data contamination or memorization.
  arXiv  Detail & Related papers  (2025-06-14T00:25:26Z)
• Long-Form Information Alignment Evaluation Beyond Atomic Facts [60.25969380388974]
  We introduce MontageLie, a benchmark that constructs deceptive narratives by "montaging" truthful statements without introducing explicit hallucinations.<n>We propose DoveScore, a novel framework that jointly verifies factual accuracy and event-order consistency.
  arXiv  Detail & Related papers  (2025-05-21T17:46:38Z)
• RelGNN: Composite Message Passing for Relational Deep Learning [56.48834369525997]
  We introduce RelGNN, a novel GNN framework specifically designed to leverage the unique structural characteristics of the graphs built from relational databases.<n>RelGNN is evaluated on 30 diverse real-world tasks from Relbench (Fey et al., 2024), and achieves state-of-the-art performance on the vast majority tasks, with improvements of up to 25%.
  arXiv  Detail & Related papers  (2025-02-10T18:58:40Z)
• Alice Benchmarks: Connecting Real World Re-Identification with the Synthetic [92.02220105679713]
  We introduce the Alice benchmarks, large-scale datasets providing benchmarks and evaluation protocols to the research community. Within the Alice benchmarks, two object re-ID tasks are offered: person and vehicle re-ID. As an important feature of our real target, the clusterability of its training set is not manually guaranteed to make it closer to a real domain adaptation test scenario.
  arXiv  Detail & Related papers  (2023-10-06T17:58:26Z)
• EALink: An Efficient and Accurate Pre-trained Framework for Issue-Commit Link Recovery [54.34661595290837]
  We propose an efficient and accurate pre-trained framework called EALink for issue-commit link recovery. We construct a large-scale dataset and conduct extensive experiments to demonstrate the power of EALink. Results show that EALink outperforms the state-of-the-art methods by a large margin (15.23%-408.65%) on various evaluation metrics.
  arXiv  Detail & Related papers  (2023-08-21T14:46:43Z)
• An Empirical Study on Data Leakage and Generalizability of Link Prediction Models for Issues and Commits [7.061740334417124]
  LinkFormer preserves and improves the accuracy of existing predictions. Our findings support that to simulate real-world scenarios effectively, researchers must maintain the temporal flow of data.
  arXiv  Detail & Related papers  (2022-11-01T10:54:26Z)
• Incorporating Relevance Feedback for Information-Seeking Retrieval using Few-Shot Document Re-Ranking [56.80065604034095]
  We introduce a kNN approach that re-ranks documents based on their similarity with the query and the documents the user considers relevant. To evaluate our different integration strategies, we transform four existing information retrieval datasets into the relevance feedback scenario.
  arXiv  Detail & Related papers  (2022-10-19T16:19:37Z)
• Bi-level Alignment for Cross-Domain Crowd Counting [113.78303285148041]
  Current methods rely on external data for training an auxiliary task or apply an expensive coarse-to-fine estimation. We develop a new adversarial learning based method, which is simple and efficient to apply. We evaluate our approach on five real-world crowd counting benchmarks, where we outperform existing approaches by a large margin.
  arXiv  Detail & Related papers  (2022-05-12T02:23:25Z)
• Automatically Generating Counterfactuals for Relation Exaction [18.740447044960796]
  relation extraction (RE) is a fundamental task in natural language processing. Current deep neural models have achieved high accuracy but are easily affected by spurious correlations. We develop a novel approach to derive contextual counterfactuals for entities.
  arXiv  Detail & Related papers  (2022-02-22T04:46:10Z)
• Automated Recovery of Issue-Commit Links Leveraging Both Textual and Non-textual Data [2.578242050187029]
  Current state-of-the-art approaches for automated commit-issue linking suffer from low precision, leading to unreliable results. We propose Hybrid-Linker to overcome such limitations by exploiting two information channels. We evaluate Hybrid-Linker against competing approaches, namely FRLink and DeepLink on a dataset of 12 projects.
  arXiv  Detail & Related papers  (2021-07-05T09:38:44Z)
• Automated Mapping of Vulnerability Advisories onto their Fix Commits in Open Source Repositories [7.629717457706326]
  We present an approach that combines practical experience and machine-learning (ML) An advisory record containing key information about a vulnerability is extracted from an advisory. A subset of candidate fix commits is obtained from the source code repository of the affected project.
  arXiv  Detail & Related papers  (2021-03-24T17:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.