How to Build AI Agents by Augmenting LLMs with Codified Human Expert Domain Knowledge? A Software Engineering Framework

• URL: http://arxiv.org/abs/2601.15153v1
• Date: Wed, 21 Jan 2026 16:23:22 GMT
• Title: How to Build AI Agents by Augmenting LLMs with Codified Human Expert Domain Knowledge? A Software Engineering Framework
• Authors: Choro Ulan uulu, Mikhail Kulyabin, Iris Fuhrmann, Jan Joosten, Nuno Miguel Martins Pacheco, Filippos Petridis, Rebecca Johnson, Jan Bosch, Helena Holmström Olsson,
• Abstract summary: Critical domain knowledge typically resides with few experts.<n>Non-experts struggle to create effective visualizations.<n>This paper investigates how to capture and embed human domain knowledge into AI agent systems.
• Score: 3.0049184484925604
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Critical domain knowledge typically resides with few experts, creating organizational bottlenecks in scalability and decision-making. Non-experts struggle to create effective visualizations, leading to suboptimal insights and diverting expert time. This paper investigates how to capture and embed human domain knowledge into AI agent systems through an industrial case study. We propose a software engineering framework to capture human domain knowledge for engineering AI agents in simulation data visualization by augmenting a Large Language Model (LLM) with a request classifier, Retrieval-Augmented Generation (RAG) system for code generation, codified expert rules, and visualization design principles unified in an agent demonstrating autonomous, reactive, proactive, and social behavior. Evaluation across five scenarios spanning multiple engineering domains with 12 evaluators demonstrates 206% improvement in output quality, with our agent achieving expert-level ratings in all cases versus baseline's poor performance, while maintaining superior code quality with lower variance. Our contributions are: an automated agent-based system for visualization generation and a validated framework for systematically capturing human domain knowledge and codifying tacit expert knowledge into AI agents, demonstrating that non-experts can achieve expert-level outcomes in specialized domains.

Related papers

• Let the Barbarians In: How AI Can Accelerate Systems Performance Research [80.43506848683633]
  We term this iterative cycle of generation, evaluation, and refinement AI-Driven Research for Systems.<n>We demonstrate that ADRS-generated solutions can match or even outperform human state-of-the-art designs.
  arXiv  Detail & Related papers  (2025-12-16T18:51:23Z)
• Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization [69.36509281190662]
  Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck.<n>We consider using AI agents to automate this manual coding, and focus on the open question of optimal agent design.<n>We demonstrate that a simple agent framework consistently generates adaptation code that outperforms human-expert solutions.
  arXiv  Detail & Related papers  (2025-12-02T18:42:26Z)
• AI as Cognitive Amplifier: Rethinking Human Judgment in the Age of Generative AI [0.65268245109828]
  I propose a three-level model of AI engagement.<n>I argue that the transition between levels requires not technical training but development of domain expertise and metacognitive skills.
  arXiv  Detail & Related papers  (2025-10-30T11:55:34Z)
• Executable Knowledge Graphs for Replicating AI Research [65.41207324831583]
  Executable Knowledge Graphs (xKG) is a modular and pluggable knowledge base that automatically integrates technical insights, code snippets, and domain-specific knowledge extracted from scientific literature.<n>Code will released at https://github.com/zjunlp/xKG.
  arXiv  Detail & Related papers  (2025-10-20T17:53:23Z)
• A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems [53.37728204835912]
  Most existing AI systems rely on manually crafted configurations that remain static after deployment.<n>Recent research has explored agent evolution techniques that aim to automatically enhance agent systems based on interaction data and environmental feedback.<n>This survey aims to provide researchers and practitioners with a systematic understanding of self-evolving AI agents.
  arXiv  Detail & Related papers  (2025-08-10T16:07:32Z)
• Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training [67.895981259683]
  General AI Agents are increasingly recognized as foundational frameworks for the next generation of artificial intelligence.<n>Current agent systems are either closed-source or heavily reliant on a variety of paid APIs and proprietary tools.<n>We present Cognitive Kernel-Pro, a fully open-source and (to the maximum extent) free multi-module agent framework.
  arXiv  Detail & Related papers  (2025-08-01T08:11:31Z)
• Can AI Master Econometrics? Evidence from Econometrics AI Agent on Expert-Level Tasks [9.52446148818128]
  We develop an Econometrics AI Agent'' built on the open-source MetaGPT framework.<n>This agent exhibits outstanding performance in: (1) planning econometric tasks strategically, (2) generating and executing code, (3) employing error-based reflection for improved robustness, and (4) allowing iterative refinement through multi-round conversations.
  arXiv  Detail & Related papers  (2025-06-01T06:34:42Z)
• On the Evaluation of Engineering Artificial General Intelligence [5.802869598386355]
  We discuss the challenges and propose a framework for evaluating engineering artificial general intelligence (eAGI) agents.<n>We consider eAGI as a specialization of artificial general intelligence (AGI)<n>eAGI agents should possess a unique blend of background knowledge (recall and retrieve) of facts and methods.
  arXiv  Detail & Related papers  (2025-05-15T18:52:47Z)
• Leveraging Multi-AI Agents for Cross-Domain Knowledge Discovery [0.0]
  This study introduces a novel approach to cross-domain knowledge discovery through the deployment of multi-AI agents. Our findings demonstrate the superior capability of domain specific multi-AI agent system in identifying and bridging knowledge gaps.
  arXiv  Detail & Related papers  (2024-04-12T14:50:41Z)
• Decision Rule Elicitation for Domain Adaptation [93.02675868486932]
  Human-in-the-loop machine learning is widely used in artificial intelligence (AI) to elicit labels from experts. In this work, we allow experts to additionally produce decision rules describing their decision-making. We show that decision rule elicitation improves domain adaptation of the algorithm and helps to propagate expert's knowledge to the AI model.
  arXiv  Detail & Related papers  (2021-02-23T08:07:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.