HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding

• URL: http://arxiv.org/abs/2505.22906v2
• Date: Fri, 30 May 2025 18:45:49 GMT
• Title: HiLDe: Intentional Code Generation via Human-in-the-Loop Decoding
• Authors: Emmanuel Anaya González, Raven Rothkopf, Sorin Lerner, Nadia Polikarpova,
• Abstract summary: We propose Human-in-the-loop Decoding, a novel interaction technique that allows users to observe and directly influence LLM decisions during code generation.<n>We implement this technique in HiLDe, a code completion assistant that highlights critical decisions made by the LLM and provides local alternatives for the user to explore.<n>In a within-subjects study (N=18) on security-related tasks, we found that HiLDe led participants to generate significantly fewer vulnerabilities and better align code generation with their goals.
• Score: 2.7884384057284612
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While AI programming tools hold the promise of increasing programmers' capabilities and productivity to a remarkable degree, they often exclude users from essential decision-making processes, causing many to effectively "turn off their brains" and over-rely on solutions provided by these systems. These behaviors can have severe consequences in critical domains, like software security. We propose Human-in-the-loop Decoding, a novel interaction technique that allows users to observe and directly influence LLM decisions during code generation, in order to align the model's output with their personal requirements. We implement this technique in HiLDe, a code completion assistant that highlights critical decisions made by the LLM and provides local alternatives for the user to explore. In a within-subjects study (N=18) on security-related tasks, we found that HiLDe led participants to generate significantly fewer vulnerabilities and better align code generation with their goals compared to a traditional code completion assistant.

Related papers

• Generative AI for Object-Oriented Programming: Writing the Right Code and Reasoning the Right Logic [2.170361965861349]
  Large language models (LLMs) have diverse applications spanning finance, commonsense knowledge graphs, medicine, and visual analysis.<n>Our work aims to address this gap by presenting a vision from the perspectives of key stakeholders involved in an OOP task.<n> Furthermore, we propose ways to augment existing logical reasoning and code writing, ultimately enhancing the programming experience.
  arXiv  Detail & Related papers  (2025-08-07T03:38:17Z)
• Large Language Model Unlearning for Source Code [65.42425213605114]
  PROD is a novel unlearning approach that enables LLMs to forget undesired code content while preserving their code generation capabilities.<n>Our evaluation demonstrates that PROD achieves superior balance between forget quality and model utility compared to existing unlearning approaches.
  arXiv  Detail & Related papers  (2025-06-20T16:27:59Z)
• Towards Machine-Generated Code for the Resolution of User Intentions [2.762180345826837]
  The advent of AI may signal a shift in user-provided intent resolution through the deployment of model-generated code.<n>In this paper, we investigate the feasibility of generating and executing through code generation that results from prompting an LLM with a concrete user intention.<n>We provide an in-depth analysis and comparison of various user intentions, the resulting code, and its execution.
  arXiv  Detail & Related papers  (2025-04-24T13:19:17Z)
• Pragmatic Reasoning improves LLM Code Generation [35.78260347663757]
  We propose CodeRSA, a novel code candidate reranking mechanism built upon the Rational Speech Act (RSA) framework.<n>We evaluate CodeRSA using one of the latest Large Language Models on a popular code generation dataset.
  arXiv  Detail & Related papers  (2025-02-20T12:44:26Z)
• Enhancing Trust in Language Model-Based Code Optimization through RLHF: A Research Design [0.0]
  This research aims to develop reliable, LM-powered methods for code optimization that effectively integrate human feedback.<n>This work aligns with the broader objectives of advancing cooperative and human-centric aspects of software engineering.
  arXiv  Detail & Related papers  (2025-02-10T18:48:45Z)
• zsLLMCode: An Effective Approach for Code Embedding via LLM with Zero-Shot Learning [6.976968804436321]
  This paper proposes a novel zero-shot approach, zsLLMCode, to generate code embeddings by using large language models (LLMs) and sentence embedding models.<n>The results have demonstrated the effectiveness and superiority of our method over state-of-the-art unsupervised approaches.
  arXiv  Detail & Related papers  (2024-09-23T01:03:15Z)
• Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making [51.737762570776006]
  LLM-ACTR is a novel neuro-symbolic architecture that provides human-aligned and versatile decision-making. Our framework extracts and embeds knowledge of ACT-R's internal decision-making process as latent neural representations. Our experiments on novel Design for Manufacturing tasks show both improved task performance as well as improved grounded decision-making capability.
  arXiv  Detail & Related papers  (2024-08-17T11:49:53Z)
• Agent-Driven Automatic Software Improvement [55.2480439325792]
  This research proposal aims to explore innovative solutions by focusing on the deployment of agents powered by Large Language Models (LLMs) The iterative nature of agents, which allows for continuous learning and adaptation, can help surpass common challenges in code generation. We aim to use the iterative feedback in these systems to further fine-tune the LLMs underlying the agents, becoming better aligned to the task of automated software improvement.
  arXiv  Detail & Related papers  (2024-06-24T15:45:22Z)
• DeAL: Decoding-time Alignment for Large Language Models [59.63643988872571]
  Large Language Models (LLMs) are nowadays expected to generate content aligned with human preferences. We propose DeAL, a framework that allows the user to customize reward functions and enables Detime Alignment of LLMs. Our experiments show that we can DeAL with fine-grained trade-offs, improve adherence to alignment objectives, and address residual gaps in LLMs.
  arXiv  Detail & Related papers  (2024-02-05T06:12:29Z)
• An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project [1.433758865948252]
  Large Language Models (LLMs) represent a leap in artificial intelligence, excelling in tasks using human language(s) In this paper, we analyze the AI-generated code, prompts used for code generation, and the human intervention levels to integrate the code into the code base. Our findings suggest that LLMs can play a crucial role in the early stages of software development.
  arXiv  Detail & Related papers  (2024-01-29T14:32:32Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [84.31119464141631]
  This work employs Large Language Models (LLMs) as a decision-making component for complex autonomous driving scenarios.<n>Extensive experiments demonstrate that our proposed method not only consistently surpasses baseline approaches in single-vehicle tasks, but also helps handle complex driving behaviors even multi-vehicle coordination.
  arXiv  Detail & Related papers  (2023-10-04T17:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.