RAILS: Retrieval-Augmented Intelligence for Learning Software Development

• URL: http://arxiv.org/abs/2506.22742v1
• Date: Sat, 28 Jun 2025 03:30:04 GMT
• Title: RAILS: Retrieval-Augmented Intelligence for Learning Software Development
• Abstract summary: Large Language Models (LLMs) like GPT-3.5-Turbo produce incomplete code or incorrect imports when lacking access to external or project-specific documentation.<n>We introduce RAILS, a framework that augments LLM prompts with semantically retrieved context from curated resources.<n>We evaluated RAILS on 78 real-world Java import error cases spanning standard libraries, GUI APIs, external tools, and custom utilities.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) like GPT-3.5-Turbo are increasingly used to assist software development, yet they often produce incomplete code or incorrect imports, especially when lacking access to external or project-specific documentation. We introduce RAILS (Retrieval-Augmented Intelligence for Learning Software Development), a framework that augments LLM prompts with semantically retrieved context from curated Java resources using FAISS and OpenAI embeddings. RAILS incorporates an iterative validation loop guided by compiler feedback to refine suggestions. We evaluated RAILS on 78 real-world Java import error cases spanning standard libraries, GUI APIs, external tools, and custom utilities. Despite using the same LLM, RAILS outperforms baseline prompting by preserving intent, avoiding hallucinations, and surfacing correct imports even when libraries are unavailable locally. Future work will integrate symbolic filtering via PostgreSQL and extend support to other languages and IDEs.

Related papers

• Teaching LLMs a Low-Resource Language: Enhancing Code Completion in Pharo [51.12236828558682]
  Large Language Models (LLMs) unlocked new possibilities in automated code writing, becoming the backbone of most code completion tools.<n>LLMs excel in mainstream languages, but they often lack support for the so-called low-resource languages where training data is scarce.<n>A concrete example is Pharo, a Smalltalk-inspired language whose IDE currently offers only single-token completion.
  arXiv  Detail & Related papers  (2026-07-06T11:11:18Z)
• No Resource, No Benchmarks, No Problem? Evaluating and Improving LLMs for Code Generation in No-Resource Languages [50.536461876371526]
  Large Language Models (LLMs) have advanced the automation of software engineering tasks.<n>Most research in this area has focused on high-resource languages, such as Python or Java, which benefit from abundant training data.<n>No-resource languages for which LLMs have seen virtually no training data remain largely unstudied.
  arXiv  Detail & Related papers  (2026-06-15T15:08:55Z)
• RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices [54.956760584923295]
  Researchers have made substantial progress using Large Language Models (LLMs) for code generation.<n>However, developers typically write code based on structured designs or specifications rather than raw natural language descriptions.<n>This gap between existing benchmarks and real industry development practices means that current benchmark scores may not accurately reflect how much code generation can help automate development tasks.
  arXiv  Detail & Related papers  (2026-04-24T15:35:54Z)
• Validated Code Translation for Projects with External Libraries [8.31687433629915]
  We present a translation and validation framework for translating Go projects with external dependencies to Rust.<n>Our approach combines (i) a retrieval mechanism that maps Go library APIs to Rust APIs, and (ii) a cross-language validation pipeline.<n>We evaluate our system on six real-world Go repositories with non-trivial external dependencies.
  arXiv  Detail & Related papers  (2026-02-20T10:55:14Z)
• LSPRAG: LSP-Guided RAG for Language-Agnostic Real-Time Unit Test Generation [19.781961858094398]
  We present LSPRAG, a framework for concise-context retrieval tailored for real-time, language-agnostic unit test generation.<n>By reusing mature Language Server Protocol (LSP) back-ends, LSPRAG provides an LLM with language-aware context retrieval.<n>Compared to the best performance of baselines, LSPRAG increased line coverage by up to 174.55% for Golang, 213.31% for Java, and 31.57% for Python.
  arXiv  Detail & Related papers  (2025-10-25T08:19:21Z)
• Automated and Context-Aware Code Documentation Leveraging Advanced LLMs [0.0]
  Existing automated approaches primarily focused on code summarization, leaving a gap in template-based documentation generation.<n>We develop a novel, context-aware dataset for Javadoc generation that includes critical structural and semantic information from modern Javas.<n>We evaluate five open-source LLMs (including LLaMA-3.1, Gemma Phi-3, Mistral, Qwen-2.5) using zero-shot, few-shot, and fine-tuned setups and provide a comparative analysis of their performance.
  arXiv  Detail & Related papers  (2025-09-16T06:27:09Z)
• How Robust are LLM-Generated Library Imports? An Empirical Study using Stack Overflow [3.076436880934678]
  We conduct an empirical study of six Large Language Models (LLMs)<n>We analyze the types of libraries they import, the characteristics of those libraries, and the extent to which the recommendations are usable out of the box.<n>Our results show that LLMs predominantly favour third-party libraries over standard ones, and often recommend mature, popular, and permissively licensed dependencies.
  arXiv  Detail & Related papers  (2025-07-14T21:35:29Z)
• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• LLMs Love Python: A Study of LLMs' Bias for Programming Languages and Libraries [15.140178992235123]
  Large Language Models (LLMs) play an increasing role in code generation.<n>This study provides the first in-depth investigation into LLM preferences for programming languages and libraries used when generating code.<n>Our findings reveal that LLMs heavily favour Python when solving language-agnostic problems.
  arXiv  Detail & Related papers  (2025-03-21T14:29:35Z)
• Codellm-Devkit: A Framework for Contextualizing Code LLMs with Program Analysis Insights [9.414198519543564]
  We present codellm-devkit (hereafter, CLDK'), an open-source library that significantly simplifies the process of performing program analysis. CLDK offers developers an intuitive and user-friendly interface, making it incredibly easy to provide rich program analysis context to code LLMs.
  arXiv  Detail & Related papers  (2024-10-16T20:05:59Z)
• CodeRAG-Bench: Can Retrieval Augment Code Generation? [78.37076502395699]
  We conduct a systematic, large-scale analysis of code generation using retrieval-augmented generation.<n>We first curate a comprehensive evaluation benchmark, CodeRAG-Bench, encompassing three categories of code generation tasks.<n>We examine top-performing models on CodeRAG-Bench by providing contexts retrieved from one or multiple sources.
  arXiv  Detail & Related papers  (2024-06-20T16:59:52Z)
• Class-Level Code Generation from Natural Language Using Iterative, Tool-Enhanced Reasoning over Repository [4.767858874370881]
  We introduce RepoClassBench, a benchmark designed to rigorously evaluate LLMs in generating class-level code within real-world repositories. RepoClassBench includes "Natural Language to Class generation" tasks across Java, Python & C# from a selection of repositories. We introduce Retrieve-Repotools-Reflect (RRR), a novel approach that equips LLMs with static analysis tools to iteratively navigate & reason about repository-level context.
  arXiv  Detail & Related papers  (2024-04-22T03:52:54Z)
• IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators [49.903001442804594]
  This work investigates the prospect of leveraging compiler intermediate representations (IR) to improve the multilingual capabilities of Code-LMs. We first compile SLTrans, a parallel dataset consisting of nearly 4M self-contained source code files. Next, we carry out continued causal language modelling training on SLTrans, forcing the Code-LMs to learn the IR language. Our resulting models, dubbed IRCoder, display sizeable and consistent gains across a wide variety of code generation tasks and metrics.
  arXiv  Detail & Related papers  (2024-03-06T17:52:08Z)
• Enhancing LLM-Based Coding Tools through Native Integration of IDE-Derived Static Context [41.91246546266515]
  We argue that Integrated Development Environments (IDEs) can provide direct, accurate and real-time cross-file information for repository-level code completion. We propose a framework that leveragesIDE native static contexts for cross-context construction and diagnosis results for self-refinement.
  arXiv  Detail & Related papers  (2024-02-06T01:59:41Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)
• LILO: Learning Interpretable Libraries by Compressing and Documenting Code [71.55208585024198]
  We introduce LILO, a neurosymbolic framework that iteratively synthesizes, compresses, and documents code. LILO combines LLM-guided program synthesis with recent algorithmic advances in automated from Stitch. We find that AutoDoc boosts performance by helping LILO's synthesizer to interpret and deploy learned abstractions.
  arXiv  Detail & Related papers  (2023-10-30T17:55:02Z)
• CodeTF: One-stop Transformer Library for State-of-the-art Code LLM [72.1638273937025]
  We present CodeTF, an open-source Transformer-based library for state-of-the-art Code LLMs and code intelligence. Our library supports a collection of pretrained Code LLM models and popular code benchmarks. We hope CodeTF is able to bridge the gap between machine learning/generative AI and software engineering.
  arXiv  Detail & Related papers  (2023-05-31T05:24:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.