AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions

• URL: http://arxiv.org/abs/2605.21082v1
• Date: Wed, 20 May 2026 12:17:43 GMT
• Title: AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions
• Abstract summary: AutoRPA is a framework that distills the decision logic of ReAct-style agents into robust RPA functions.<n> Experiments show that RPA functions generated by AutoRPA successfully solve similar tasks while reducing token usage by 82% to 96%.
• Score: 27.06451670215353
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Model (LLM) based agents have demonstrated proficiency in multi-step interactions with graphical user interfaces (GUIs). While most research focuses on improving single-task performance, practical scenarios often involve repetitive GUI tasks for which invoking LLM reasoning repeatedly, i.e., the ReAct paradigm, is inefficient. Prior to LLMs, traditional Robotic Process Automation (RPA) offers runtime efficiency but demands significant manual effort to develop and maintain. To bridge this gap, we propose AutoRPA, a framework that automatically distills the decision logic of ReAct-style agents into robust RPA functions. AutoRPA introduces two core innovations: (1) A translator-builder pipeline, where a translator agent converts hard-coded ReAct actions into soft-coded procedures, and a builder agent synthesizes robust RPA functions via retrieval-augmented generation over multiple trajectories; (2) A hybrid repair strategy during code verification, combining RPA execution with ReAct-based fallback for iterative refinement. Experiments across multiple GUI environments demonstrate that RPA functions generated by AutoRPA successfully solve similar tasks while reducing token usage by 82% to 96%, significantly improving runtime efficiency and reusability.

Related papers

• EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience Learning [24.190399395358014]
  We formulate MCP-GUI as a unified hybrid policy learning problem where the agent learns when each modality provides complementary advantages.<n>We propose a self-evolving framework with a fully automatic pipeline that orchestrates automatic environment generation and validation, trajectory collection, gap-driven task synthesis, and quality-filtered training.
  arXiv  Detail & Related papers  (2026-04-10T18:46:05Z)
• GPA: Learning GUI Process Automation from Demonstrations [59.82483926336919]
  GPA is a lightweight but general vision-based Robotic Process Automation (RPA)<n>It enables fast and stable process replay with only a single demo.<n>It addresses the fragility of traditional RPA and the non-deterministic risks of current vision language model-based GUI agents.
  arXiv  Detail & Related papers  (2026-04-02T06:27:52Z)
• ComAgent: Multi-LLM based Agentic AI Empowered Intelligent Wireless Networks [62.031889234230725]
  6G networks rely on complex cross-layer optimization.<n> manually translating high-level intents into mathematical formulations remains a bottleneck.<n>We present ComAgent, a multi-LLM agentic AI framework.
  arXiv  Detail & Related papers  (2026-01-27T13:43:59Z)
• A Nascent Taxonomy of Machine Learning in Intelligent Robotic Process Automation [0.0]
  We conduct a literature review to explore the connections between RPA and machine learning.<n>Our taxonomy comprises the two meta-characteristics RPA-ML integration and RPA-ML interaction.
  arXiv  Detail & Related papers  (2025-09-19T08:01:27Z)
• Are LLM Agents the New RPA? A Comparative Study with RPA Across Enterprise Workflows [0.0]
  AACU enables intelligent agents to perform tasks through natural language instructions and autonomous interaction with user interfaces.<n>This study investigates whether AACU can serve as a viable alternative to RPA in enterprise workflow automation.
  arXiv  Detail & Related papers  (2025-09-04T13:22:44Z)
• CoAct-1: Computer-using Agents with Coding as Actions [94.99657662893338]
  CoAct-1 is a novel multi-agent system that combines GUI-based control with direct programmatic execution.<n>We evaluate our system on the challenging OSWorld benchmark, where CoAct-1 achieves a new state-of-the-art success rate of 60.76%.
  arXiv  Detail & Related papers  (2025-08-05T21:33:36Z)
• ERPA: Efficient RPA Model Integrating OCR and LLMs for Intelligent Document Processing [0.0]
  This paper presents ERPA, an innovative Robotic Process Automation (RPA) model designed to enhance ID data extraction and optimize Optical Character Recognition (OCR) tasks within immigration.<n> Benchmark comparisons demonstrate that ERPA significantly reduces processing times by up to 94 percent, completing ID data extraction in just 9.94 seconds.
  arXiv  Detail & Related papers  (2024-12-24T09:44:43Z)
• LMRPA: Large Language Model-Driven Efficient Robotic Process Automation for OCR [0.0]
  This paper introduces LMRPA, a novel Large Model-Driven Robotic Process Automation model.<n>It is designed to greatly improve the efficiency and speed of Optical Character Recognition (OCR) tasks.
  arXiv  Detail & Related papers  (2024-12-24T00:21:36Z)
• AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML [56.565200973244146]
  Automated machine learning (AutoML) accelerates AI development by automating tasks in the development pipeline.<n>Recent works have started exploiting large language models (LLM) to lessen such burden.<n>This paper proposes AutoML-Agent, a novel multi-agent framework tailored for full-pipeline AutoML.
  arXiv  Detail & Related papers  (2024-10-03T20:01:09Z)
• AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning [54.47116888545878]
  AutoAct is an automatic agent learning framework for QA. It does not rely on large-scale annotated data and synthetic planning trajectories from closed-source models.
  arXiv  Detail & Related papers  (2024-01-10T16:57:24Z)
• Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives [92.0321404272942]
  Reinforcement learning can be used to build general-purpose robotic systems. However, training RL agents to solve robotics tasks still remains challenging. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance.
  arXiv  Detail & Related papers  (2021-10-28T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.