When to (or not to) trust intelligent machines: Insights from an
evolutionary game theory analysis of trust in repeated games
- URL: http://arxiv.org/abs/2007.11338v1
- Date: Wed, 22 Jul 2020 10:53:49 GMT
- Title: When to (or not to) trust intelligent machines: Insights from an
evolutionary game theory analysis of trust in repeated games
- Authors: The Anh Han, Cedric Perret and Simon T. Powers
- Abstract summary: We study the viability of trust-based strategies in repeated games.
These are reciprocal strategies that cooperate as long as the other player is observed to be cooperating.
By doing so, they reduce the opportunity cost of verifying whether the action of their co-player was actually cooperative.
- Score: 0.8701566919381222
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The actions of intelligent agents, such as chatbots, recommender systems, and
virtual assistants are typically not fully transparent to the user.
Consequently, using such an agent involves the user exposing themselves to the
risk that the agent may act in a way opposed to the user's goals. It is often
argued that people use trust as a cognitive shortcut to reduce the complexity
of such interactions. Here we formalise this by using the methods of
evolutionary game theory to study the viability of trust-based strategies in
repeated games. These are reciprocal strategies that cooperate as long as the
other player is observed to be cooperating. Unlike classic reciprocal
strategies, once mutual cooperation has been observed for a threshold number of
rounds they stop checking their co-player's behaviour every round, and instead
only check with some probability. By doing so, they reduce the opportunity cost
of verifying whether the action of their co-player was actually cooperative. We
demonstrate that these trust-based strategies can outcompete strategies that
are always conditional, such as Tit-for-Tat, when the opportunity cost is
non-negligible. We argue that this cost is likely to be greater when the
interaction is between people and intelligent agents, because of the reduced
transparency of the agent. Consequently, we expect people to use trust-based
strategies more frequently in interactions with intelligent agents. Our results
provide new, important insights into the design of mechanisms for facilitating
interactions between humans and intelligent agents, where trust is an essential
factor.
Related papers
- Fast Peer Adaptation with Context-aware Exploration [63.08444527039578]
We propose a peer identification reward for learning agents in multi-agent games.
This reward motivates the agent to learn a context-aware policy for effective exploration and fast adaptation.
We evaluate our method on diverse testbeds that involve competitive (Kuhn Poker), cooperative (PO-Overcooked), or mixed (Predator-Prey-W) games with peer agents.
arXiv Detail & Related papers (2024-02-04T13:02:27Z) - ProAgent: Building Proactive Cooperative Agents with Large Language
Models [89.53040828210945]
ProAgent is a novel framework that harnesses large language models to create proactive agents.
ProAgent can analyze the present state, and infer the intentions of teammates from observations.
ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various coordination scenarios.
arXiv Detail & Related papers (2023-08-22T10:36:56Z) - Game Theoretic Rating in N-player general-sum games with Equilibria [26.166859475522106]
We propose novel algorithms suitable for N-player, general-sum rating of strategies in normal-form games according to the payoff rating system.
This enables well-established solution concepts, such as equilibria, to be leveraged to efficiently rate strategies in games with complex strategic interactions.
arXiv Detail & Related papers (2022-10-05T12:33:03Z) - Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
This work proposes a novel reinforcement learning mechanism based on the social impact of rivalry behavior.
Our proposed model aggregates objective and social perception mechanisms to derive a rivalry score that is used to modulate the learning of artificial agents.
arXiv Detail & Related papers (2022-08-22T14:06:06Z) - Safe adaptation in multiagent competition [48.02377041620857]
In multiagent competitive scenarios, ego-agents may have to adapt to new opponents with previously unseen behaviors.
As the ego-agent updates its own behavior to exploit the opponent, its own behavior could become more exploitable.
We develop a safe adaptation approach in which the ego-agent is trained against a regularized opponent model.
arXiv Detail & Related papers (2022-03-14T23:53:59Z) - Cooperative Artificial Intelligence [0.0]
We argue that there is a need for research on the intersection between game theory and artificial intelligence.
We discuss the problem of how an external agent can promote cooperation between artificial learners.
We show that the resulting cooperative outcome is stable in certain games even if the planning agent is turned off.
arXiv Detail & Related papers (2022-02-20T16:50:37Z) - Incorporating Rivalry in Reinforcement Learning for a Competitive Game [65.2200847818153]
This study focuses on providing a novel learning mechanism based on a rivalry social impact.
Based on the concept of competitive rivalry, our analysis aims to investigate if we can change the assessment of these agents from a human perspective.
arXiv Detail & Related papers (2020-11-02T21:54:18Z) - Learning to Incentivize Other Learning Agents [73.03133692589532]
We show how to equip RL agents with the ability to give rewards directly to other agents, using a learned incentive function.
Such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games.
Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.
arXiv Detail & Related papers (2020-06-10T20:12:38Z) - Learning to Resolve Alliance Dilemmas in Many-Player Zero-Sum Games [22.38765498549914]
We argue that a systematic study of many-player zero-sum games is a crucial element of artificial intelligence research.
Using symmetric zero-sum matrix games, we demonstrate formally that alliance formation may be seen as a social dilemma.
We show how reinforcement learning may be augmented with a peer-to-peer contract mechanism to discover and enforce alliances.
arXiv Detail & Related papers (2020-02-27T10:32:31Z) - Multi-Issue Bargaining With Deep Reinforcement Learning [0.0]
This paper evaluates the use of deep reinforcement learning in bargaining games.
Two actor-critic networks were trained for the bidding and acceptance strategy.
Neural agents learn to exploit time-based agents, achieving clear transitions in decision preference values.
They also demonstrate adaptive behavior against different combinations of concession, discount factors, and behavior-based strategies.
arXiv Detail & Related papers (2020-02-18T18:33:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.