AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback

• URL: http://arxiv.org/abs/2402.01469v2
• Date: Fri, 25 Oct 2024 09:30:32 GMT
• Title: AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback
• Abstract summary: We present AMOR, an agent framework based on open-source large language models (LLMs) AMOR builds reasoning logic over a finite state machine (FSM) that solves problems through autonomous executions and transitions over disentangled modules. Based on this reasoning and feedback framework, we develop AMOR through two-stage fine-tuning: warm-up and adaptation.
• Score: 91.76169903959376
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The notable success of large language models (LLMs) has sparked an upsurge in building language agents to complete various complex tasks. We present AMOR, an agent framework based on open-source LLMs, which reasons with external knowledge bases and adapts to specific domains through human supervision to the reasoning process. AMOR builds reasoning logic over a finite state machine (FSM) that solves problems through autonomous executions and transitions over disentangled modules. This allows humans to provide direct feedback to the individual modules, and thus naturally forms process supervision. Based on this reasoning and feedback framework, we develop AMOR through two-stage fine-tuning: warm-up and adaptation. The former fine-tunes the LLM with examples automatically constructed from various public datasets, enabling AMOR to generalize across different knowledge environments, while the latter tailors AMOR to specific domains using process feedback. Extensive experiments across multiple domains demonstrate the advantage of AMOR to strong baselines, thanks to its FSM-based reasoning and process feedback mechanism. The code and data are publicly available at \url{https://github.com/JianGuanTHU/AMOR}.

Related papers

• Bridging the Agent-World Gap: Text World Models for LLM-based Agents [83.8237017249529]
  Large language model (LLM)-based agents are increasingly used in interactive textual environments.<n>Many remain largely reactive, mapping observations to actions without an explicit model of how these environments are structured and evolve.<n>This motivates text world models (TWMs): transition models over textual states that, given a state and a candidate action, predict the resulting webpage, terminal output, API response, or user reply.<n>We systematically review text world models for LLM-based agents, organized around a formal framework and the agent lifecycle.
  arXiv  Detail & Related papers  (2026-06-08T04:58:52Z)
• DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation [10.633307460813523]
  DomAgent is an autonomous coding agent for large language models (LLMs)<n>DomRetriever is a novel retrieval module that emulates how humans learn domain-specific knowledge.<n> Experimental results show that DomAgent significantly enhances domain-specific code generation.
  arXiv  Detail & Related papers  (2026-03-22T22:39:32Z)
• FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents [25.60249598832918]
  FT-Dojo is an interactive environment comprising 13 tasks across 5 domains.<n>We develop FT-Agent, an autonomous system that mirrors human experts by leveraging evaluation-driven feedback.
  arXiv  Detail & Related papers  (2026-03-02T10:37:11Z)
• ReCreate: Reasoning and Creating Domain Agents Driven by Experience [14.353866611611672]
  ReCreate is an experience-driven framework for the automatic creation of domain agents.<n>We introduce an agent-as-optimizer paradigm that effectively learns from experience.<n>In experiments across diverse domains, ReCreate consistently outperforms human-designed agents.
  arXiv  Detail & Related papers  (2026-01-16T09:00:03Z)
• From What to Why: Thought-Space Recommendation with Small Language Models [2.134948383299948]
  Large Language Models (LLMs) have advanced recommendation capabilities through enhanced reasoning, but pose significant challenges for real-world deployment.<n>Small Language Models (SLMs) offer an efficient alternative, their reasoning capabilities for recommendation remain underexplored.<n>We propose PULSE (Preference Understanding by Latent Semantic Embeddings), a framework that treats SLM-generated rationales as director learning signals.
  arXiv  Detail & Related papers  (2025-10-08T11:22:26Z)
• A Concurrent Modular Agent: Framework for Autonomous LLM Agents [0.995321385692873]
  We introduce the Concurrent Modular Agent (CMA), a framework that orchestrates multiple Large-Language-Model (LLM)-based modules.<n>We consider this approach to be a practical realization of Minsky's Society of Mind theory.<n>The emergent properties observed in our system suggest that complex cognitive phenomena like self-awareness may indeed arise from the organized interaction of simpler processes.
  arXiv  Detail & Related papers  (2025-08-26T13:58:31Z)
• BAR: A Backward Reasoning based Agent for Complex Minecraft Tasks [15.48158268901061]
  To complete a task, a large language model (LLM) based agent needs to decompose it into easily executed steps by planning.<n>Existing studies mainly conduct the planning by inferring what steps should be executed next starting from the agent's initial state.<n>We propose to study this issue in Minecraft, a virtual environment that simulates complex tasks based on real-world scenarios.
  arXiv  Detail & Related papers  (2025-05-20T08:35:35Z)
• SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy Segment Optimization [70.11167263638562]
  Social relation reasoning aims to identify relation categories such as friends, spouses, and colleagues from images. We first present a simple yet well-crafted framework named name, which combines the perception capability of Vision Foundation Models (VFMs) and the reasoning capability of Large Language Models (LLMs) within a modular framework.
  arXiv  Detail & Related papers  (2024-10-28T18:10:26Z)
• The Compressor-Retriever Architecture for Language Model OS [20.56093501980724]
  This paper explores the concept of using a language model as the core component of an operating system (OS) A key challenge in realizing such an LM OS is managing the life-long context and ensuring statefulness across sessions. We introduce compressor-retriever, a model-agnostic architecture designed for life-long context management.
  arXiv  Detail & Related papers  (2024-09-02T23:28:15Z)
• Hello Again! LLM-powered Personalized Agent for Long-term Dialogue [63.65128176360345]
  We introduce a model-agnostic framework, the Long-term Dialogue Agent (LD-Agent) It incorporates three independently tunable modules dedicated to event perception, persona extraction, and response generation. The effectiveness, generality, and cross-domain capabilities of LD-Agent are empirically demonstrated.
  arXiv  Detail & Related papers  (2024-06-09T21:58:32Z)
• Enhancing Open-Domain Task-Solving Capability of LLMs via Autonomous Tool Integration from GitHub [79.31134731122462]
  We introduce OpenAct benchmark to evaluate the open-domain task-solving capability, built on human expert consultation and repositories in GitHub.<n>We present OpenAgent, a novel LLM-based agent system that can tackle evolving queries in open domains through autonomously integrating specialized tools from GitHub.
  arXiv  Detail & Related papers  (2023-12-28T15:47:30Z)
• Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendations [50.81844184210381]
  We propose a general paradigm that augments large language models with DOmain-specific KnowledgE to enhance their performance on practical applications, namely DOKE. This paradigm relies on a domain knowledge extractor, working in three steps: 1) preparing effective knowledge for the task; 2) selecting the knowledge for each specific sample; and 3) expressing the knowledge in an LLM-understandable way.
  arXiv  Detail & Related papers  (2023-11-16T07:09:38Z)
• Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning [79.53130089003986]
  Large Language Models (LLMs) have become a feasible solution for handling tasks in various domains. In this paper, we introduce how to fine-tune a LLM model that can be privately deployed for content moderation.
  arXiv  Detail & Related papers  (2023-10-05T09:09:44Z)
• External Reasoning: Towards Multi-Large-Language-Models Interchangeable Assistance with Human Feedback [0.0]
  This paper proposes that Large Language Models (LLMs) could be augmented through the selective integration of knowledge from external repositories. Central to this approach is the establishment of a tiered policy for textbfExternal Reasoning based on Multiple LLM Interchange Assistance. The results indicate state-of-the-art performance in crefcomparison, surpassing existing solutions including ChatPDF.com.
  arXiv  Detail & Related papers  (2023-07-05T17:05:32Z)
• Inner Monologue: Embodied Reasoning through Planning with Language Models [81.07216635735571]
  Large Language Models (LLMs) can be applied to domains beyond natural language processing. LLMs planning in embodied environments need to consider not just what skills to do, but also how and when to do them. We propose that by leveraging environment feedback, LLMs are able to form an inner monologue that allows them to more richly process and plan in robotic control scenarios.
  arXiv  Detail & Related papers  (2022-07-12T15:20:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.