Counterfactual Planning in AGI Systems

• URL: http://arxiv.org/abs/2102.00834v1
• Date: Fri, 29 Jan 2021 13:44:14 GMT
• Title: Counterfactual Planning in AGI Systems
• Authors: Koen Holtman
• Abstract summary: Key step in counterfactual planning is to use an AGI machine learning system to construct a counterfactual world model. A counterfactual planning agent determines the action that best maximizes expected utility in this counterfactual planning world. We use counterfactual planning to construct an AGI agent emergency stop button, and a safety interlock that will automatically stop the agent before it undergoes an intelligence explosion.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present counterfactual planning as a design approach for creating a range of safety mechanisms that can be applied in hypothetical future AI systems which have Artificial General Intelligence. The key step in counterfactual planning is to use an AGI machine learning system to construct a counterfactual world model, designed to be different from the real world the system is in. A counterfactual planning agent determines the action that best maximizes expected utility in this counterfactual planning world, and then performs the same action in the real world. We use counterfactual planning to construct an AGI agent emergency stop button, and a safety interlock that will automatically stop the agent before it undergoes an intelligence explosion. We also construct an agent with an input terminal that can be used by humans to iteratively improve the agent's reward function, where the incentive for the agent to manipulate this improvement process is suppressed. As an example of counterfactual planning in a non-agent AGI system, we construct a counterfactual oracle. As a design approach, counterfactual planning is built around the use of a graphical notation for defining mathematical counterfactuals. This two-diagram notation also provides a compact and readable language for reasoning about the complex types of self-referencing and indirect representation which are typically present inside machine learning agents.

Related papers

• Ask-before-Plan: Proactive Language Agents for Real-World Planning [68.08024918064503]
  Proactive Agent Planning requires language agents to predict clarification needs based on user-agent conversation and agent-environment interaction. We propose a novel multi-agent framework, Clarification-Execution-Planning (textttCEP), which consists of three agents specialized in clarification, execution, and planning.
  arXiv  Detail & Related papers  (2024-06-18T14:07:28Z)
• Automated Process Planning Based on a Semantic Capability Model and SMT [50.76251195257306]
  In research of manufacturing systems and autonomous robots, the term capability is used for a machine-interpretable specification of a system function. We present an approach that combines these two topics: starting from a semantic capability model, an AI planning problem is automatically generated.
  arXiv  Detail & Related papers  (2023-12-14T10:37:34Z)
• Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty [56.30846158280031]
  Task planning for embodied AI has been one of the most challenging problems. We propose a task-agnostic method named 'planning as in-painting' The proposed framework achieves promising performances in various embodied AI tasks.
  arXiv  Detail & Related papers  (2023-12-02T10:07:17Z)
• On Computing Universal Plans for Partially Observable Multi-Agent Path Finding [11.977931648859176]
  We argue that it is beneficial to formulate multi-agent routing problems as universal planning problems. We implement a system called ASP-MAUPF (Answer Set Programming for Multi-Agent Universal Plan Finding) for computing them.
  arXiv  Detail & Related papers  (2023-05-25T16:06:48Z)
• Online Grounding of PDDL Domains by Acting and Sensing in Unknown Environments [62.11612385360421]
  This paper proposes a framework that allows an agent to perform different tasks. We integrate machine learning models to abstract the sensory data, symbolic planning for goal achievement and path planning for navigation. We evaluate the proposed method in accurate simulated environments, where the sensors are RGB-D on-board camera, GPS and compass.
  arXiv  Detail & Related papers  (2021-12-18T21:48:20Z)
• A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learning [104.3643447579578]
  We present an end-to-end, model-based deep reinforcement learning agent which dynamically attends to relevant parts of its state. The design allows agents to learn to plan effectively, by attending to the relevant objects, leading to better out-of-distribution generalization.
  arXiv  Detail & Related papers  (2021-06-03T19:35:19Z)
• Knowledge-Based Hierarchical POMDPs for Task Planning [0.34998703934432684]
  The main goal in task planning is to build a sequence of actions that takes an agent from an initial state to a goal state. In robotics, this is particularly difficult because actions usually have several possible results, and sensors are prone to produce measurements with error. We present a scheme to encode knowledge about the robot and its environment, that promotes the modularity and reuse of information.
  arXiv  Detail & Related papers  (2021-03-19T05:45:05Z)
• Software Architecture for Next-Generation AI Planning Systems [0.0]
  We propose a service-oriented planning architecture to be at the core of the ability to design, develop and use next-generation AI planning systems. We incorporate software design principles and patterns into the architecture to allow for usability, interoperability and reusability of the planning capabilities.
  arXiv  Detail & Related papers  (2021-02-22T13:43:45Z)
• Modelling Multi-Agent Epistemic Planning in ASP [66.76082318001976]
  This paper presents an implementation of a multi-shot Answer Set Programming-based planner that can reason in multi-agent epistemic settings. The paper shows how the planner, exploiting an ad-hoc epistemic state representation and the efficiency of ASP solvers, has competitive performance results on benchmarks collected from the literature.
  arXiv  Detail & Related papers  (2020-08-07T06:35:56Z)
• AGI Agent Safety by Iteratively Improving the Utility Function [0.0]
  We present an AGI safety layer that creates a special dedicated input terminal to support the iterative improvement of an AGI agent's utility function. We show ongoing work in mapping it to a Causal Influence Diagram (CID) We then present the design of a learning agent, a design that wraps the safety layer around either a known machine learning system, or a potential future AGI-level learning system.
  arXiv  Detail & Related papers  (2020-07-10T14:30:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.