Rulebook: bringing co-routines to reinforcement learning environments

• URL: http://arxiv.org/abs/2504.19625v1
• Date: Mon, 28 Apr 2025 09:34:34 GMT
• Title: Rulebook: bringing co-routines to reinforcement learning environments
• Authors: Massimo Fioravanti, Samuele Pasini, Giovanni Agosta,
• Abstract summary: Rulebook is a compiled language designed to automatically generate the state machine required to interact with machine learning algorithms.<n>It allows users to express programs without needing to be aware of the specific interface required by the ML components.
• Score: 0.688204255655161
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Reinforcement learning (RL) algorithms, due to their reliance on external systems to learn from, require digital environments (e.g., simulators) with very simple interfaces, which in turn constrain significantly the implementation of such environments. In particular, these environments are implemented either as separate processes or as state machines, leading to synchronization and communication overheads in the first case, and to unstructured programming in the second. We propose a new domain-specific, co-routine-based, compiled language, called Rulebook, designed to automatically generate the state machine required to interact with machine learning (ML) algorithms and similar applications, with no performance overhead. Rulebook allows users to express programs without needing to be aware of the specific interface required by the ML components. By decoupling the execution model of the program from the syntactical encoding of the program, and thus without the need for manual state management, Rulebook allows to create larger and more sophisticated environments at a lower development cost.

Related papers

• Skill Discovery for Software Scripting Automation via Offline Simulations with LLMs [63.10710876536337]
  We propose an offline simulation framework to curate a software-specific skillset, a collection of verified scripts. Our framework comprises two components: (1) task creation, using top-down functionality and bottom-up API synergy exploration to generate helpful tasks. Experiments with Adobe Illustrator demonstrate that our framework significantly improves automation success rates, reduces response time, and saves runtime token costs.
  arXiv  Detail & Related papers  (2025-04-29T04:03:37Z)
• Model-Driven Rapid Prototyping for Control Algorithms with the GIPS Framework (System Description) [0.0]
  We have created the GIPS (Graph-Based ILP Problem Specification) framework to support rapid prototyping of software systems.<n>Developers can use our high-level specification language GIPSL (Graph-Based ILP Problem Specification Language) to specify their desired model optimization as sets of constraints and objectives.<n>GIPS is able to derive executable (Java) software artifacts automatically that optimize a given input graph instance at runtime.
  arXiv  Detail & Related papers  (2025-03-26T11:52:52Z)
• Promptware Engineering: Software Engineering for LLM Prompt Development [22.788377588087894]
  Large Language Models (LLMs) are increasingly integrated into software applications, with prompts serving as the primary 'programming' interface.<n>As a result, a new software paradigm, promptware, has emerged, using natural language prompts to interact with LLMs.<n>Unlike traditional software, which relies on formal programming languages and deterministic runtime environments, promptware is based on ambiguous, unstructured, and context-dependent natural language.
  arXiv  Detail & Related papers  (2025-03-04T08:43:16Z)
• Asynchronous Tool Usage for Real-Time Agents [61.3041983544042]
  We introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting. This work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.
  arXiv  Detail & Related papers  (2024-10-28T23:57:19Z)
• Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs [42.31298987176411]
  We introduce ROBO-INSTRUCT, which synthesizes task-specific simulation environments on the fly during program execution.<n>ROBO-INSTRUCT integrates an LLM-aided post-processing procedure to refine instructions for better alignment with robot programs.
  arXiv  Detail & Related papers  (2024-05-30T15:47:54Z)
• RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs [32.01139974519813]
  We present RedCoast, a tool crafted to automate distributed training and inference for large language models (LLMs) We also propose a mechanism that allows for the customization of diverse ML pipelines through the definition of merely three functions. As a result, Redco implementations exhibit significantly fewer lines of code compared to their official counterparts.
  arXiv  Detail & Related papers  (2023-10-25T04:32:35Z)
• mlirSynth: Automatic, Retargetable Program Raising in Multi-Level IR using Program Synthesis [48.01697184432969]
  mlirSynth translates programs from lower-level MLIR dialects to high-level ones without manually defined rules. We demonstrate its effectiveness reviby raising C programs to two distinct high-level MLIR dialects, which enables us to use existing high-level dialect specific compilation flows.
  arXiv  Detail & Related papers  (2023-10-06T12:21:50Z)
• QParallel: Explicit Parallelism for Programming Quantum Computers [62.10004571940546]
  We present a language extension for parallel quantum programming. QParallel removes ambiguities concerning parallelism in current quantum programming languages. We introduce a tool that guides programmers in the placement of parallel regions by identifying the subroutines that profit most from parallelization.
  arXiv  Detail & Related papers  (2022-10-07T16:35:16Z)
• Procedures as Programs: Hierarchical Control of Situated Agents through Natural Language [81.73820295186727]
  We propose a formalism of procedures as programs, a powerful yet intuitive method of representing hierarchical procedural knowledge for agent command and control. We instantiate this framework on the IQA and ALFRED datasets for NL instruction following.
  arXiv  Detail & Related papers  (2021-09-16T20:36:21Z)
• Composition Machines: Programming Self-Organising Software Models for the Emergence of Sequential Program Spaces [0.0]
  We propose an abstract machine, called the composition machine, which allows the definition and the execution of such models. Unlike typical abstract machines, our proposal does not compute individual programs but enables the emergence of multiple programs at once.
  arXiv  Detail & Related papers  (2021-08-11T18:39:47Z)
• How could Neural Networks understand Programs? [67.4217527949013]
  It is difficult to build a model to better understand programs, by either directly applying off-the-shelf NLP pre-training techniques to the source code, or adding features to the model by theshelf. We propose a novel program semantics learning paradigm, that the model should learn from information composed of (1) the representations which align well with the fundamental operations in operational semantics, and (2) the information of environment transition.
  arXiv  Detail & Related papers  (2021-05-10T12:21:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.