Skilled AI Agents for Embedded and IoT Systems Development

• URL: http://arxiv.org/abs/2603.19583v1
• Date: Fri, 20 Mar 2026 02:56:15 GMT
• Title: Skilled AI Agents for Embedded and IoT Systems Development
• Abstract summary: We introduce a skills-based agentic framework for HIL embedded development together with IoT-SkillsBench.<n> IoT-SkillsBench is a benchmark designed to systematically evaluate AI agents in real embedded programming environments.<n>We show that concise human-expert skills with structured expert knowledge enable near-perfect success rates across platforms.
• Score: 28.432934288413975
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) and agentic systems have shown promise for automated software development, but applying them to hardware-in-the-loop (HIL) embedded and Internet-of-Things (IoT) systems remains challenging due to the tight coupling between software logic and physical hardware behavior. Code that compiles successfully may still fail when deployed on real devices because of timing constraints, peripheral initialization requirements, or hardware-specific behaviors. To address this challenge, we introduce a skills-based agentic framework for HIL embedded development together with IoT-SkillsBench, a benchmark designed to systematically evaluate AI agents in real embedded programming environments. IoT-SkillsBench spans three representative embedded platforms, 23 peripherals, and 42 tasks across three difficulty levels, where each task is evaluated under three agent configurations (no-skills, LLM-generated skills, and human-expert skills) and validated through real hardware execution. Across 378 hardware validated experiments, we show that concise human-expert skills with structured expert knowledge enable near-perfect success rates across platforms.

Related papers

• StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows [63.57092302641323]
  We introduce textbfStartupBench, an E2E agent benchmark grounded in market-validated AI startup products.<n>We study AI products with demonstrated adoption, together with their product and users, to identify real-world tasks for which AI has established practical demand.<n>Our results reveal that many market-validated agents remain beyond the reliable capabilities of current general-purpose agents.
  arXiv  Detail & Related papers  (2026-08-18T14:01:32Z)
• Building AI That Works: ESnet's Pragmatic Approach to AI-Driven Operational Excellence [0.909937273720124]
  ORBIT is an agentic AI system integrated into ServiceNow, ESnet's primary incident management platform.<n>The design uses a modular, layered architecture comprising a centralized reasoning hub, access via MCPs for ESnet data sources, a semantic search layer, and an operator-facing chat interface.<n>Key results show that ORBIT successfully delivered all six initial tasks, and the architecture enabled rapid development of two additional tasks proposed by NOC engineers.
  arXiv  Detail & Related papers  (2026-07-24T23:20:59Z)
• SkillNet: Create, Evaluate, and Connect AI Skills [159.47504178122156]
  SkillNet is an open infrastructure designed to create, evaluate, and organize AI skills at scale.<n>Our infrastructure integrates a repository of over 200,000 skills, an interactive platform, and a versatile Python toolkit.
  arXiv  Detail & Related papers  (2026-02-26T14:24:02Z)
• Agentic Pipelines in Embedded Software Engineering: Emerging Practices and Challenges [2.0769172070951067]
  A new transformation is underway in software engineering, driven by the rapid adoption of generative AI in development.<n>For embedded software engineering organizations, however, this marks their first experience integrating AI into safety-critical and resource-constrained environments.<n>The strict demands for determinism, reliability, and traceability pose unique challenges for adopting generative technologies.
  arXiv  Detail & Related papers  (2026-01-15T09:30:46Z)
• AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities [8.086360127362815]
  Large language model (LLM)-based coding agents autonomously plan, execute, and interact with tools such as compilers, debuggers, and version control systems.<n>Unlike conventional code generation, these agents decompose goals, coordinate multi-step processes, and adapt based on feedback, reshaping software development practices.
  arXiv  Detail & Related papers  (2025-08-15T00:14:31Z)
• Verbal Process Supervision Elicits Better Coding Agents [0.9558392439655016]
  This work introduces CURA, a code understanding and reasoning agent system enhanced with verbal process supervision (VPS)<n>CURA achieves a 3.65% improvement over baseline models on challenging benchmarks like BigCodeBench.
  arXiv  Detail & Related papers  (2025-03-24T09:48:59Z)
• Develop AI Agents for System Engineering in Factorio [1.5824959429406713]
  This position paper advocates for training and evaluating AI agents' system engineering abilities through automation-oriented sandbox games.<n>By directing research efforts in this direction, we can equip AI agents with the specialized reasoning and long-horizon planning necessary to design, maintain, and optimize tomorrow's most demanding engineering projects.
  arXiv  Detail & Related papers  (2025-02-03T16:26:17Z)
• EmbedGenius: Towards Automated Software Development for Generic Embedded IoT Systems [11.524778651869044]
  This paper introduces EmbedGenius, the first fully automated software development platform for general-purpose embedded IoT systems.<n>The key idea is to leverage the reasoning ability of Large Language Models (LLMs) and embedded system expertise to automate the hardware-in-the-loop development process.<n>We evaluate EmbedGenius's performance across 71 modules and four mainstream embedded development platforms with over 350 IoT tasks.
  arXiv  Detail & Related papers  (2024-12-12T08:34:12Z)
• Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence [79.5316642687565]
  Existing multi-agent frameworks often struggle with integrating diverse capable third-party agents. We propose the Internet of Agents (IoA), a novel framework that addresses these limitations. IoA introduces an agent integration protocol, an instant-messaging-like architecture design, and dynamic mechanisms for agent teaming and conversation flow control.
  arXiv  Detail & Related papers  (2024-07-09T17:33:24Z)
• OS-Copilot: Towards Generalist Computer Agents with Self-Improvement [48.29860831901484]
  We introduce OS-Copilot, a framework to build generalist agents capable of interfacing with comprehensive elements in an operating system (OS) We use OS-Copilot to create FRIDAY, a self-improving embodied agent for automating general computer tasks. On GAIA, a general AI assistants benchmark, FRIDAY outperforms previous methods by 35%, showcasing strong generalization to unseen applications via accumulated skills from previous tasks.
  arXiv  Detail & Related papers  (2024-02-12T07:29:22Z)
• Technology Readiness Levels for Machine Learning Systems [107.56979560568232]
  Development and deployment of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. We have developed a proven systems engineering approach for machine learning development and deployment. Our "Machine Learning Technology Readiness Levels" framework defines a principled process to ensure robust, reliable, and responsible systems.
  arXiv  Detail & Related papers  (2021-01-11T15:54:48Z)
• Technology Readiness Levels for AI & ML [79.22051549519989]
  Development of machine learning systems can be executed easily with modern tools, but the process is typically rushed and means-to-an-end. Engineering systems follow well-defined processes and testing standards to streamline development for high-quality, reliable results. We propose a proven systems engineering approach for machine learning development and deployment.
  arXiv  Detail & Related papers  (2020-06-21T17:14:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.