SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution

• URL: http://arxiv.org/abs/2608.18933v1
• Date: Wed, 19 Aug 2026 14:01:13 GMT
• Title: SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution
• Abstract summary: Large language model (LLM) based agents have demonstrated remarkable proficiency in automated software issue resolution.<n>They often struggle to resolve issues in a specific repository because they lack project-specific knowledge.<n>We propose SkillForge, a framework that proactively acquires project-specific knowledge from the repository itself.
• Score: 21.5992590228819
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model (LLM) based agents have demonstrated remarkable proficiency in automated software issue resolution, yet they often struggle to resolve issues in a specific repository because they lack project-specific knowledge. Existing self-evolving approaches acquire such knowledge from repository history or online repair trajectories, but they either depend on available historical issue-resolution signals or incur substantial per-issue test-time exploration cost. In this paper, we propose SkillForge, a self-distillation framework that proactively acquires project-specific knowledge from the repository itself. Instead of waiting for real issues to expose project-specific knowledge gaps, SkillForge synthesizes project-specific issues by re-implementing test-covered core functionalities of the repository. By resolving these synthetic issues, SkillForge distills reusable project-specific knowledge into entity-grounded skills and associates them with relevant repository entities for future issue resolution. Extensive experiments using both open-source and closed-source models show that SkillForge consistently improves issue resolution performance over strong baselines. These results demonstrate that proactively acquiring project-specific knowledge before solving real issues substantially improves downstream software issue resolution.

Related papers

• Know Before Fix: QA-Driven Repository Knowledge Acquisition for Software Issue Resolution [14.539105528943958]
  ACQUIRE is a QA-driven framework for software issue resolution.<n>Mirroring how developers first comprehend unfamiliar code before attempting a fix, ACQUIRE explicitly acquires repository knowledge prior to repair.
  arXiv  Detail & Related papers  (2026-07-13T05:41:01Z)
• From Knowing to Acting: Benchmarking Self-Awareness Capability of LLM Agents [69.07911499189129]
  We introduce KAPRO, a framework that evaluates cognitive-behavioral alignment by decoupling an agent's metacognitive judgment (Knowing) from its spontaneous execution (Acting)<n>Experiments show that self-awareness capability is strongly correlated with task success but degrades sharply in internal-capability settings.<n> proprietary and reasoning-oriented models demonstrate more reliable cognitive gating.
  arXiv  Detail & Related papers  (2026-06-09T17:17:08Z)
• RepoMirage: Probing Repository Context Reasoning in Code Agents with Perturbations [51.43574078961796]
  We introduce RepoMirage, a two-stage evaluation suite built on SWE-Bench Verified.<n>RepoMirage-Perturb applies three types of semantics-preserving repository-level perturbations, revealing a clear performance drop when correct solving requires broader context access.<n>We propose RepoAnchor, a structure-first prototype workflow that separates repository exploration from downstream problem solving.
  arXiv  Detail & Related papers  (2026-05-25T06:26:43Z)
• BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing? [61.247730037229815]
  We introduce BeyondSWE, a comprehensive benchmark that broadens existing evaluations along two axes - resolution scope and knowledge scope.<n>To investigate the role of external knowledge, we develop SearchSWE, a framework that integrates deep search with coding abilities.<n>This work offers both a realistic, challenging evaluation benchmark and a flexible framework to advance research toward more capable code agents.
  arXiv  Detail & Related papers  (2026-03-03T17:52:01Z)
• NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents [79.29376673236142]
  Existing benchmarks fail to rigorously evaluate the long-horizon capabilities required to build complete software systems.<n>We present NL2Repo Bench, a benchmark explicitly designed to evaluate the long-horizon repository generation ability of coding agents.
  arXiv  Detail & Related papers  (2025-12-14T15:12:13Z)
• Predicting Abandonment of Open Source Software Projects with An Integrated Feature Framework [15.50732865972961]
  Open Source Software (OSS) is a cornerstone of contemporary software development, yet the increasing prevalence of OSS project abandonment threatens global supply chains.<n>In this work, we address these challenges by reliably detecting OSS project abandonment through a dual approach: explicit archival status and rigorous semantic analysis of project documentation or description.<n>Building on this foundation, we introduce an integrated, multi-perspective feature framework for abandonment prediction, capturing user-centric, maintainer-centric, and project evolution features.<n>Our work provides reproducible, scalable, and practitioner-oriented support for understanding and managing abandonment risk in large OSS ecosystems.
  arXiv  Detail & Related papers  (2025-07-29T10:45:24Z)
• Towards an Interpretable Analysis for Estimating the Resolution Time of Software Issues [1.4039240369201997]
  We build an issue monitoring system that extracts the actual effort required to fix issues on a per-project basis.<n>Our approach employs topic modeling to capture issue semantics and leverages metadata for interpretable resolution time analysis.
  arXiv  Detail & Related papers  (2025-05-02T08:38:59Z)
• QualiTagger: Automating software quality detection in issue trackers [4.917423556150366]
  This research uses cutting edge models like Transformers to identify what text is usually associated with different quality properties.<n>We also study the distribution of such qualities in issue trackers from openly accessible software repositories.
  arXiv  Detail & Related papers  (2025-04-15T10:40:40Z)
• Knowledge Islands: Visualizing Developers Knowledge Concentration [0.0]
  Knowledge Islands is a tool that visualizes the concentration of knowledge in a software repository using a state-of-the-art knowledge model. It enables practitioners to analyze GitHub projects, determine where knowledge is concentrated, and implement measures to maintain project health.
  arXiv  Detail & Related papers  (2024-08-16T13:32:49Z)
• Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models [49.23348672822087]
  We propose Knowledge Crosswords, a benchmark consisting of incomplete knowledge networks bounded by structured factual constraints. The novel setting of geometric knowledge reasoning necessitates new LM abilities beyond existing atomic/linear multi-hop QA. We conduct extensive experiments to evaluate existing LLMs and approaches on Knowledge Crosswords.
  arXiv  Detail & Related papers  (2023-10-02T15:43:53Z)
• A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA [67.75989848202343]
  This paper presents a unified end-to-end retriever-reader framework towards knowledge-based VQA. We shed light on the multi-modal implicit knowledge from vision-language pre-training models to mine its potential in knowledge reasoning. Our scheme is able to not only provide guidance for knowledge retrieval, but also drop these instances potentially error-prone towards question answering.
  arXiv  Detail & Related papers  (2022-06-30T02:35:04Z)
• KILT: a Benchmark for Knowledge Intensive Language Tasks [102.33046195554886]
  We present a benchmark for knowledge-intensive language tasks (KILT) All tasks in KILT are grounded in the same snapshot of Wikipedia. We find that a shared dense vector index coupled with a seq2seq model is a strong baseline.
  arXiv  Detail & Related papers  (2020-09-04T15:32:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.