From Helpful to Trustworthy: LLM Agents for Pair Programming

• URL: http://arxiv.org/abs/2604.10300v1
• Date: Sat, 11 Apr 2026 17:39:57 GMT
• Title: From Helpful to Trustworthy: LLM Agents for Pair Programming
• Abstract summary: LLM-based coding agents are increasingly used to generate code, tests, and documentation.<n>Still, their outputs can be plausible yet misaligned with developer intent and provide limited evidence for review in evolving projects.<n>This research proposes a systematic study of multi-agent LLM pair programming that externalizes intent and uses development tools for iterative validation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM-based coding agents are increasingly used to generate code, tests, and documentation. Still, their outputs can be plausible yet misaligned with developer intent and provide limited evidence for review in evolving projects. This limits our understanding of how to structure LLM pair-programming workflows so that artifacts remain reliable, auditable, and maintainable over time. To address this gap, this doctoral research proposes a systematic study of multi-agent LLM pair programming that externalizes intent and uses development tools for iterative validation. The plan includes three studies: translating informal problem statements into standards aligned requirements and formal specifications; refining tests and implementations using automated feedback, such as solver-backed counterexamples; and supporting maintenance tasks, including refactoring, API migrations, and documentation updates, while preserving validated behavior. The expected outcome is a clearer understanding of when multi-agent workflows increase trust, along with practical guidance for building reliable programming assistants for real-world development.

Related papers

• RealBench: A Repo-Level Code Generation Benchmark Aligned with Real-World Software Development Practices [54.956760584923295]
  Researchers have made substantial progress using Large Language Models (LLMs) for code generation.<n>However, developers typically write code based on structured designs or specifications rather than raw natural language descriptions.<n>This gap between existing benchmarks and real industry development practices means that current benchmark scores may not accurately reflect how much code generation can help automate development tasks.
  arXiv  Detail & Related papers  (2026-04-24T15:35:54Z)
• Prompt-Driven Code Summarization: A Systematic Literature Review [4.238464304669881]
  Large language models (LLMs) offer a promising solution by automatically generating natural language descriptions from source code.<n>This systematic review literature consolidates existing evidence, categorizes prompting paradigms, examines their effectiveness, and identifies gaps to guide future research and practical adoption.
  arXiv  Detail & Related papers  (2026-04-16T03:25:28Z)
• Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgement [8.059802912761919]
  We uncover a systematic failure of large language models (LLMs) in matching code to natural language requirements.<n>More detailed prompt design, particularly with those requiring explanations and proposed corrections, leads to higher misjudgment rates.<n>We propose a Fix-guided Verification Filter that treats the model proposed fix as executable counterfactual evidence.
  arXiv  Detail & Related papers  (2026-02-28T08:35:25Z)
• Understanding Specification-Driven Code Generation with LLMs: An Empirical Study Design [2.687678248171195]
  Large Language Models (LLMs) are increasingly integrated into software development, yet their behavior in structured, specification-driven processes remains poorly understood.<n>This paper presents an empirical study design using CURRANTE, a Visual Studio Code extension that enables a human-in-the-loop workflow for LLM-assisted code generation.<n>The study aims to analyze how human intervention in specification and test refinement influences the quality and dynamics of LLM-generated code.
  arXiv  Detail & Related papers  (2026-01-07T12:46:57Z)
• On LLM-Assisted Generation of Smart Contracts from Business Processes [0.08192907805418582]
  Large language models (LLMs) have changed the reality of how software is produced.<n>We present an exploratory study to investigate the use of LLMs for generating smart contract code from business process descriptions.<n>Our results show that LLM performance falls short of the perfect reliability required for smart contract development.
  arXiv  Detail & Related papers  (2025-07-30T20:39:45Z)
• Do AI models help produce verified bug fixes? [62.985237003585674]
  Large Language Models are used to produce corrections to software bugs.<n>This paper investigates how programmers use Large Language Models to complement their own skills.<n>The results are a first step towards a proper role for AI and LLMs in providing guaranteed-correct fixes to program bugs.
  arXiv  Detail & Related papers  (2025-07-21T17:30:16Z)
• Training Language Models to Generate Quality Code with Program Analysis Feedback [66.0854002147103]
  Code generation with large language models (LLMs) is increasingly adopted in production but fails to ensure code quality.<n>We propose REAL, a reinforcement learning framework that incentivizes LLMs to generate production-quality code.
  arXiv  Detail & Related papers  (2025-05-28T17:57:47Z)
• AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios [51.46347732659174]
  Large Language Models (LLMs) have demonstrated advanced capabilities in real-world agentic applications.<n>AgentIF is the first benchmark for systematically evaluating LLM instruction following ability in agentic scenarios.
  arXiv  Detail & Related papers  (2025-05-22T17:31:10Z)
• CodeIF: Benchmarking the Instruction-Following Capabilities of Large Language Models for Code Generation [20.013757490442064]
  We introduce CodeIF, the first benchmark designed to assess the abilities of Large Language Models (LLMs) to adhere to task-oriented instructions.<n>CodeIF encompasses a broad range of tasks, including function synthesis, algorithmic instructions, and code explanation.<n>We conduct extensive experiments with LLMs, analyzing their strengths and limitations in meeting the demands of these tasks.
  arXiv  Detail & Related papers  (2025-02-26T14:19:49Z)
• Experiential Co-Learning of Software-Developing Agents [83.34027623428096]
  Large language models (LLMs) have brought significant changes to various domains, especially in software development. We introduce Experiential Co-Learning, a novel LLM-agent learning framework. Experiments demonstrate that the framework enables agents to tackle unseen software-developing tasks more effectively.
  arXiv  Detail & Related papers  (2023-12-28T13:50:42Z)
• ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code [76.84199699772903]
  ML-Bench is a benchmark rooted in real-world programming applications that leverage existing code repositories to perform tasks. To evaluate both Large Language Models (LLMs) and AI agents, two setups are employed: ML-LLM-Bench for assessing LLMs' text-to-code conversion within a predefined deployment environment, and ML-Agent-Bench for testing autonomous agents in an end-to-end task execution within a Linux sandbox environment.
  arXiv  Detail & Related papers  (2023-11-16T12:03:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.