Story Shaping: Teaching Agents Human-like Behavior with Stories
- URL: http://arxiv.org/abs/2301.10107v1
- Date: Tue, 24 Jan 2023 16:19:09 GMT
- Title: Story Shaping: Teaching Agents Human-like Behavior with Stories
- Authors: Xiangyu Peng, Christopher Cui, Wei Zhou, Renee Jia, Mark Riedl
- Abstract summary: We introduce Story Shaping, in which a reinforcement learning agent infers tacit knowledge from an exemplar story of how to accomplish a task.
An intrinsic reward is generated based on the similarity between the agent's inferred world state graph and the inferred story world graph.
We conducted experiments in text-based games requiring commonsense reasoning and shaping the behaviors of agents as virtual game characters.
- Score: 9.649246837532417
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Reward design for reinforcement learning agents can be difficult in
situations where one not only wants the agent to achieve some effect in the
world but where one also cares about how that effect is achieved. For example,
we might wish for an agent to adhere to a tacit understanding of commonsense,
align itself to a preference for how to behave for purposes of safety, or
taking on a particular role in an interactive game. Storytelling is a mode for
communicating tacit procedural knowledge. We introduce a technique, Story
Shaping, in which a reinforcement learning agent infers tacit knowledge from an
exemplar story of how to accomplish a task and intrinsically rewards itself for
performing actions that make its current environment adhere to that of the
inferred story world. Specifically, Story Shaping infers a knowledge graph
representation of the world state from observations, and also infers a
knowledge graph from the exemplar story. An intrinsic reward is generated based
on the similarity between the agent's inferred world state graph and the
inferred story world graph. We conducted experiments in text-based games
requiring commonsense reasoning and shaping the behaviors of agents as virtual
game characters.
Related papers
- Generating Visual Stories with Grounded and Coreferent Characters [63.07511918366848]
We present the first model capable of predicting visual stories with consistently grounded and coreferent character mentions.
Our model is finetuned on a new dataset which we build on top of the widely used VIST benchmark.
We also propose new evaluation metrics to measure the richness of characters and coreference in stories.
arXiv Detail & Related papers (2024-09-20T14:56:33Z) - StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning [8.851718319632973]
Large Language Models (LLMs) drive the behavior of virtual characters, allowing plots to emerge from interactions between characters and their environments.
We propose a novel plot creation workflow that mediates between a writer's authorial intent and the emergent behaviors from LLM-driven character simulation.
The process creates "living stories" that dynamically adapt to various game world states, resulting in narratives co-created by the author, character simulation, and player.
arXiv Detail & Related papers (2024-05-17T23:04:51Z) - SCO-VIST: Social Interaction Commonsense Knowledge-based Visual
Storytelling [12.560014305032437]
This paper introduces SCO-VIST, a framework representing the image sequence as a graph with objects and relations.
SCO-VIST then takes this graph representing plot points and creates bridges between plot points with semantic and occurrence-based edge weights.
This weighted story graph produces the storyline in a sequence of events using Floyd-Warshall's algorithm.
arXiv Detail & Related papers (2024-02-01T04:09:17Z) - Synthesizing Physical Character-Scene Interactions [64.26035523518846]
It is necessary to synthesize such interactions between virtual characters and their surroundings.
We present a system that uses adversarial imitation learning and reinforcement learning to train physically-simulated characters.
Our approach takes physics-based character motion generation a step closer to broad applicability.
arXiv Detail & Related papers (2023-02-02T05:21:32Z) - Infusing Commonsense World Models with Graph Knowledge [89.27044249858332]
We study the setting of generating narratives in an open world text adventure game.
A graph representation of the underlying game state can be used to train models that consume and output both grounded graph representations and natural language descriptions and actions.
arXiv Detail & Related papers (2023-01-13T19:58:27Z) - Learning Action-Effect Dynamics from Pairs of Scene-graphs [50.72283841720014]
We propose a novel method that leverages scene-graph representation of images to reason about the effects of actions described in natural language.
Our proposed approach is effective in terms of performance, data efficiency, and generalization capability compared to existing models.
arXiv Detail & Related papers (2022-12-07T03:36:37Z) - Commonsense Knowledge from Scene Graphs for Textual Environments [6.617487928813374]
We investigate the advantage of employing commonsense reasoning obtained from visual datasets such as scene graph datasets.
Our proposed methods have higher and competitive performance than existing state-of-the-art methods.
arXiv Detail & Related papers (2022-10-19T03:09:17Z) - Incorporating Commonsense Knowledge into Story Ending Generation via
Heterogeneous Graph Networks [16.360265861788253]
We propose a Story Heterogeneous Graph Network (SHGN) to explicitly model both the information of story context at different levels and the multi-grained interactive relations among them.
In detail, we consider commonsense knowledge, words and sentences as three types of nodes.
We design two auxiliary tasks to implicitly capture the sentiment trend and key events lie in the context.
arXiv Detail & Related papers (2022-01-29T09:33:11Z) - Inherently Explainable Reinforcement Learning in Natural Language [14.117921448623342]
We focus on the task of creating a reinforcement learning agent that is inherently explainable.
This Hierarchically Explainable Reinforcement Learning agent operates in Interactive Fictions, text-based game environments.
Our agent is designed to treat explainability as a first-class citizen.
arXiv Detail & Related papers (2021-12-16T14:24:35Z) - PlotMachines: Outline-Conditioned Generation with Dynamic Plot State
Tracking [128.76063992147016]
We present PlotMachines, a neural narrative model that learns to transform an outline into a coherent story by tracking the dynamic plot states.
In addition, we enrich PlotMachines with high-level discourse structure so that the model can learn different writing styles corresponding to different parts of the narrative.
arXiv Detail & Related papers (2020-04-30T17:16:31Z) - Learning Dynamic Belief Graphs to Generalize on Text-Based Games [55.59741414135887]
Playing text-based games requires skills in processing natural language and sequential decision making.
In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text.
arXiv Detail & Related papers (2020-02-21T04:38:37Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.