Answer-Set-Programming-based Abstractions for Reinforcement Learning

• URL: http://arxiv.org/abs/2605.31444v1
• Date: Fri, 29 May 2026 15:40:10 GMT
• Title: Answer-Set-Programming-based Abstractions for Reinforcement Learning
• Abstract summary: Reinforcement Learning (RL) enables autonomous agents to learn, but problems often involve making learning and generalisation challenging.<n>CARCASS framework by Martijn van Otterlo demonstrates how logical approximation can model Markov Decision Processes (MDs) in first-order representations.<n>We explore ASP, which is a rich World, contrary to Prolog, fully declarative modelling language, to realise CARCASS abstractions.
• Score: 11.527421282223946
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement Learning (RL) enables autonomous agents to learn policies from experience, but realistic problems often involve enormous state spaces, making learning and generalisation challenging. Abstraction and approximation are therefore essential. Relational Reinforcement Learning (RRL) offers a way to reason about objects and their relations, and the CARCASS framework by Martijn van Otterlo demonstrates how logical representations can model Markov Decision Processes (MDPs) in first-order domains. Originally implemented in Prolog, CARCASS leverages domain knowledge to create powerful abstractions. We explore Answer-Set Programming (ASP), which is a rich and, contrary to Prolog, fully declarative modelling language, to realise CARCASS abstractions. We evaluate our ASP-based implementation in case studies of two domains, viz. Blocks World and Minigrid. Our results indicate that CARCASS with ASP provides a promising approach to constructing abstractions for RL, especially when domain knowledge is available.

Related papers

• Understanding Automated Web GUI Testing: An Empirical Study Across Exploration Strategies and State Abstractions [12.068986139687908]
  We study how exploration strategies and state abstractions jointly affect testing effectiveness.<n>No single strategy excels across all dimensions; instead, categories exhibit complementary strengths in code coverage, state coverage, and failure discovery.
  arXiv  Detail & Related papers  (2026-06-15T12:40:12Z)
• Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents [54.73952501784257]
  Humans learn abstractions and use them to plan efficiently to quickly generalize across tasks.<n>We introduce TheoryCoder-2, a new large language model (LLM) agent that actively learns reusable abstractions.<n>We conduct experiments on diverse environments, including BabyAI, Minihack and VGDL games like Sokoban.
  arXiv  Detail & Related papers  (2026-01-31T23:01:51Z)
• Using Large Language Models for Abstraction of Planning Domains - Extended Version [6.021787236982658]
  We model the agent's concrete behaviors in PDDL and investigate the use of in-context learning with large language models (LLMs)<n>We consider three categories of abstractions: abstraction of choice of alternative concrete actions, abstraction of sequences of concrete actions, and abstraction of action/predicate parameters.<n>The generated abstract PDDL domains and problem instances are then checked by symbolic validation tools as well as human experts.
  arXiv  Detail & Related papers  (2025-10-23T06:27:03Z)
• Integrating Domain Knowledge into Process Discovery Using Large Language Models [3.7448613209842967]
  We propose an interactive framework that incorporates domain knowledge, expressed in natural language, into the process discovery pipeline.<n>The framework coordinates interactions among the Large Language Models (LLMs), domain experts, and a set of backend services.<n>Our empirical study includes a case study based on a real-life event log with the involvement of domain experts, who assessed the usability and effectiveness of the framework.
  arXiv  Detail & Related papers  (2025-10-08T15:59:11Z)
• RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems [98.98963933669751]
  We train models to be capable of proposing multiple abstractions given a problem, followed by RL that incentivizes building a solution.<n>This results in a two-player RL training paradigm, abbreviated as RLAD, that jointly trains an abstraction generator and a solution generator.<n>We show that allocating more test-time compute to generating abstractions is more beneficial for performance than generating more solutions at large test budgets.
  arXiv  Detail & Related papers  (2025-10-02T17:44:23Z)
• Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains [92.36624674516553]
  Reinforcement learning with verifiable rewards (RLVR) has demonstrated significant success in enhancing mathematical reasoning and coding performance of large language models (LLMs)<n>We investigate the effectiveness and scalability of RLVR across diverse real-world domains including medicine, chemistry, psychology, economics, and education.<n>We utilize a generative scoring technique that yields soft, model-based reward signals to overcome limitations posed by binary verifications.
  arXiv  Detail & Related papers  (2025-03-31T08:22:49Z)
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning [87.30285670315334]
  textbfR1-Searcher is a novel two-stage outcome-based RL approach designed to enhance the search capabilities of Large Language Models.<n>Our framework relies exclusively on RL, without requiring process rewards or distillation for a cold start.<n>Our experiments demonstrate that our method significantly outperforms previous strong RAG methods, even when compared to the closed-source GPT-4o-mini.
  arXiv  Detail & Related papers  (2025-03-07T17:14:44Z)
• Reasoning Language Models: A Blueprint [16.04440875855868]
  Reasoning language models (RLMs) have redefined AI's problem-solving capabilities.<n>Yet, their high costs, proprietary nature, and complex architectures present accessibility and scalability challenges.<n>We propose a comprehensive blueprint that organizes RLM components into a modular framework.
  arXiv  Detail & Related papers  (2025-01-20T02:16:19Z)
• Multi-step Inference over Unstructured Data [2.169874047093392]
  High-stakes decision-making tasks in fields such as medical, legal and finance require a level of precision, comprehensiveness, and logical consistency. We have developed a neuro-symbolic AI platform to tackle these problems. The platform integrates fine-tuned LLMs for knowledge extraction and alignment with a robust symbolic reasoning engine.
  arXiv  Detail & Related papers  (2024-06-26T00:00:45Z)
• Building Minimal and Reusable Causal State Abstractions for Reinforcement Learning [63.58935783293342]
  Causal Bisimulation Modeling (CBM) is a method that learns the causal relationships in the dynamics and reward functions for each task to derive a minimal, task-specific abstraction. CBM's learned implicit dynamics models identify the underlying causal relationships and state abstractions more accurately than explicit ones.
  arXiv  Detail & Related papers  (2024-01-23T05:43:15Z)
• Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendations [50.81844184210381]
  We propose a general paradigm that augments large language models with DOmain-specific KnowledgE to enhance their performance on practical applications, namely DOKE. This paradigm relies on a domain knowledge extractor, working in three steps: 1) preparing effective knowledge for the task; 2) selecting the knowledge for each specific sample; and 3) expressing the knowledge in an LLM-understandable way.
  arXiv  Detail & Related papers  (2023-11-16T07:09:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.