Joint Learning of Experiential Rules and Policies for Large Language Model Agents

• URL: http://arxiv.org/abs/2606.27136v1
• Date: Thu, 25 Jun 2026 15:11:02 GMT
• Title: Joint Learning of Experiential Rules and Policies for Large Language Model Agents
• Abstract summary: Joint Learning of Experiential Rules and Policies for LLM Agents (JERP)<n>JERP updates a long-term experiential-rule pool and the policy from the same interaction trajectories.<n>Experiments on AlfWorld and WebShop show that JERP yields consistent gains in decision performance for complex interactive tasks.
• Score: 5.144286041118895
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For LLM agents in multi-step interactive environments, a key challenge is to make effective use of accumulated interaction experience. Existing work has typically separated two uses of such experience: keeping it outside the model as natural-language rules for later prompting, or using trajectories and feedback to update the model parameters. The former is easy to interpret but can fall out of sync with the evolving policy; the latter improves the policy more broadly but provides only limited correction for local mistakes in sparse-reward settings. We present Joint Learning of Experiential Rules and Policies for LLM Agents (JERP), which updates a long-term experiential-rule pool and the policy from the same interaction trajectories. At decision time, JERP retrieves task-relevant rules and conditions the agent on them together with the interaction history. After each episode, it uses the collected trajectories both to optimize the policy and to revise the rule pool by comparing current rollouts with reference successful trajectories. This coupling keeps the rule pool aligned with the evolving policy while allowing stable and effective behaviors to be gradually absorbed into the model itself. Experiments on AlfWorld and WebShop show that JERP yields consistent gains in decision performance for complex interactive tasks.

Related papers

• LLM-Based Hierarchical Coordinated Control with Continuation-Aware Policy Learning [6.69561560301489]
  We propose a hierarchical framework for coordinating multiple interacting units in complex engineering systems.<n>We introduce Continuation-Aware GRPO to capture the consequences of coordination decisions over subsequent control intervals.<n>We validate the framework on multi-ramp traffic control and virtual power plant (VPP) energy management, using simplified system models for training and more realistic simulators for evaluation.
  arXiv  Detail & Related papers  (2026-08-15T05:00:02Z)
• From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents [56.187590188698955]
  Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful.<n>Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles.<n>We introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories.<n>We propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment.
  arXiv  Detail & Related papers  (2026-08-03T12:52:04Z)
• TAPO: Transition-Aware Policy Optimization for LLM Agents [27.95488836373643]
  TAPO is a unified training framework that alternates between policy optimization and transition supervision.<n>It serves as a computationally lightweight, plug-and-play enhancement module for existing agent RL algorithms.<n>We conduct systematic experiments on WebShop and ALFWorld, integrating foundation models of various scales with different policy optimization algorithms.
  arXiv  Detail & Related papers  (2026-07-30T10:17:55Z)
• ARdena: Scenario-driven control of real-time LLM agents [0.0]
  This paper introduces layered scenario-driven LLM control, a framework that enables runtime behavior control through structured prompting.<n>The framework is implemented in ARDena, a real-time multimodal embodied agent that integrates speech interaction, visual perception, tool use, and avatar-based response generation.
  arXiv  Detail & Related papers  (2026-06-26T08:58:33Z)
• Communication Policy Evolution for Proactive LLM Agents [58.4960889232199]
  This paper formalizes Communication Policy, establishes textual and UI-based policies, and then evaluates communication policies across diverse environments.<n>We propose Communication Policy Evolution (CPE), a self-evolution framework for refining communication policies through rollout and prompt-level evolving.
  arXiv  Detail & Related papers  (2026-06-12T09:54:01Z)
• Learning to Learn-at-Test-Time: Language Agents with Learnable Adaptation Policies [48.31565681022125]
  Test-Time Learning (TTL) enables language agents to iteratively refine their performance through repeated interactions with the environment at inference time.<n>We argue that optimal adaptation policies should be learned from task environments, not hand-engineered based on human intuition.<n>We introduce Meta-TTL, a framework that formulates the discovery of effective adaptation policies as a bi-level optimization problem.
  arXiv  Detail & Related papers  (2026-04-01T12:41:01Z)
• Policy-Conditioned Policies for Multi-Agent Task Solving [53.67744322553693]
  In this work, we propose a paradigm shift that bridges the gap by representing policies as human-interpretable source code.<n>We reformulate the learning problem by utilizing Large Language Models (LLMs) as approximate interpreters.<n>We formalize this process as textitProgrammatic Iterated Best Response (PIBR), an algorithm where the policy code is optimized by textual gradients.
  arXiv  Detail & Related papers  (2025-12-24T07:42:10Z)
• Offline Multi-agent Reinforcement Learning via Score Decomposition [51.23590397383217]
  offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts.<n>This work is the first work to explicitly address the distributional gap between offline and online MARL.
  arXiv  Detail & Related papers  (2025-05-09T11:42:31Z)
• Imitation Learning of Correlated Policies in Stackelberg Games [17.026813111994443]
  Stackelberg games involve asymmetric interactions where a leader's strategy drives follower responses.<n>In multi-agent systems, agent behaviors are interdependent, and traditional Multi-Agent Imitation Learning (MAIL) methods often fail to capture these complex interactions.<n>We propose a correlated policy occupancy measure specifically designed for Stackelberg games and introduce the Latent Stackelberg Differential Network (LSDN) to match it.
  arXiv  Detail & Related papers  (2025-03-11T20:52:56Z)
• Let Offline RL Flow: Training Conservative Agents in the Latent Space of Normalizing Flows [58.762959061522736]
  offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. We build upon recent works on learning policies in latent action spaces and use a special form of Normalizing Flows for constructing a generative model. We evaluate our method on various locomotion and navigation tasks, demonstrating that our approach outperforms recently proposed algorithms.
  arXiv  Detail & Related papers  (2022-11-20T21:57:10Z)
• Mutual Information Regularized Offline Reinforcement Learning [76.05299071490913]
  We propose a novel MISA framework to approach offline RL from the perspective of Mutual Information between States and Actions in the dataset. We show that optimizing this lower bound is equivalent to maximizing the likelihood of a one-step improved policy on the offline dataset. We introduce 3 different variants of MISA, and empirically demonstrate that tighter mutual information lower bound gives better offline RL performance.
  arXiv  Detail & Related papers  (2022-10-14T03:22:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.