Value Engineering for Autonomous Agents
- URL: http://arxiv.org/abs/2302.08759v1
- Date: Fri, 17 Feb 2023 08:52:15 GMT
- Title: Value Engineering for Autonomous Agents
- Authors: Nieves Montes, Nardine Osman, Carles Sierra, Marija Slavkovik
- Abstract summary: Previous approaches have treated values as labels associated with some actions or states of the world, rather than as integral components of agent reasoning.
We propose a new AMA paradigm grounded in moral and social psychology, where values are instilled into agents as context-dependent goals.
We argue that this type of normative reasoning, where agents are endowed with an understanding of norms' moral implications, leads to value-awareness in autonomous agents.
- Score: 3.6130723421895947
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Machine Ethics (ME) is concerned with the design of Artificial Moral Agents
(AMAs), i.e. autonomous agents capable of reasoning and behaving according to
moral values. Previous approaches have treated values as labels associated with
some actions or states of the world, rather than as integral components of
agent reasoning. It is also common to disregard that a value-guided agent
operates alongside other value-guided agents in an environment governed by
norms, thus omitting the social dimension of AMAs. In this blue sky paper, we
propose a new AMA paradigm grounded in moral and social psychology, where
values are instilled into agents as context-dependent goals. These goals
intricately connect values at individual levels to norms at a collective level
by evaluating the outcomes most incentivized by the norms in place. We argue
that this type of normative reasoning, where agents are endowed with an
understanding of norms' moral implications, leads to value-awareness in
autonomous agents. Additionally, this capability paves the way for agents to
align the norms enforced in their societies with respect to the human values
instilled in them, by complementing the value-based reasoning on norms with
agreement mechanisms to help agents collectively agree on the best set of norms
that suit their human values. Overall, our agent model goes beyond the
treatment of values as inert labels by connecting them to normative reasoning
and to the social functionalities needed to integrate value-aware agents into
our modern hybrid human-computer societies.
Related papers
- Learning and Sustaining Shared Normative Systems via Bayesian Rule
Induction in Markov Games [2.307051163951559]
We build learning agents that cooperate flexibly with the human institutions they are embedded in.
By assuming shared norms, a newly introduced agent can infer the norms of an existing population from observations of compliance and violation.
Since agents can bootstrap common knowledge of the norms, this leads the norms to be widely adhered to, enabling new entrants to rapidly learn those norms.
arXiv Detail & Related papers (2024-02-20T21:58:40Z) - Towards Responsible AI in Banking: Addressing Bias for Fair
Decision-Making [69.44075077934914]
"Responsible AI" emphasizes the critical nature of addressing biases within the development of a corporate culture.
This thesis is structured around three fundamental pillars: understanding bias, mitigating bias, and accounting for bias.
In line with open-source principles, we have released Bias On Demand and FairView as accessible Python packages.
arXiv Detail & Related papers (2024-01-13T14:07:09Z) - Agent Alignment in Evolving Social Norms [65.45423591744434]
We propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent.
In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation.
We show that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks.
arXiv Detail & Related papers (2024-01-09T15:44:44Z) - Measuring Value Alignment [12.696227679697493]
This paper introduces a novel formalism to quantify the alignment between AI systems and human values.
By utilizing this formalism, AI developers and ethicists can better design and evaluate AI systems to ensure they operate in harmony with human values.
arXiv Detail & Related papers (2023-12-23T12:30:06Z) - Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards
and Ethical Behavior in the MACHIAVELLI Benchmark [61.43264961005614]
We develop a benchmark of 134 Choose-Your-Own-Adventure games containing over half a million rich, diverse scenarios.
We evaluate agents' tendencies to be power-seeking, cause disutility, and commit ethical violations.
Our results show that agents can both act competently and morally, so concrete progress can be made in machine ethics.
arXiv Detail & Related papers (2023-04-06T17:59:03Z) - Modeling Moral Choices in Social Dilemmas with Multi-Agent Reinforcement
Learning [4.2050490361120465]
A bottom-up learning approach may be more appropriate for studying and developing ethical behavior in AI agents.
We present a systematic analysis of the choices made by intrinsically-motivated RL agents whose rewards are based on moral theories.
We analyze the impact of different types of morality on the emergence of cooperation, defection or exploitation.
arXiv Detail & Related papers (2023-01-20T09:36:42Z) - Socially Intelligent Genetic Agents for the Emergence of Explicit Norms [0.0]
We address the emergence of explicit norms by developing agents who provide and reason about explanations for norm violations.
These agents use a genetic algorithm to produce norms and reinforcement learning to learn the values of these norms.
We find that applying explanations leads to norms that provide better cohesion and goal satisfaction for the agents.
arXiv Detail & Related papers (2022-08-07T18:48:48Z) - How Different Groups Prioritize Ethical Values for Responsible AI [75.40051547428592]
Private companies, public sector organizations, and academic groups have outlined ethical values they consider important for responsible AI technologies.
While their recommendations converge on a set of central values, little is known about the values a more representative public would find important for the AI technologies they interact with and might be affected by.
We conducted a survey examining how individuals perceive and prioritize responsible AI values across three groups.
arXiv Detail & Related papers (2022-05-16T14:39:37Z) - Aligning to Social Norms and Values in Interactive Narratives [89.82264844526333]
We focus on creating agents that act in alignment with socially beneficial norms and values in interactive narratives or text-based games.
We introduce the GALAD agent that uses the social commonsense knowledge present in specially trained language models to contextually restrict its action space to only those actions that are aligned with socially beneficial values.
arXiv Detail & Related papers (2022-05-04T09:54:33Z) - Noe: Norms Emergence and Robustness Based on Emotions in Multiagent
Systems [0.0]
This paper investigates how modeling emotions affect the emergence and robustness of social norms via social simulation experiments.
We find that an ability in agents to consider emotional responses to the outcomes of norm satisfaction and violation promote norm compliance.
arXiv Detail & Related papers (2021-04-30T14:42:22Z) - Aligning AI With Shared Human Values [85.2824609130584]
We introduce the ETHICS dataset, a new benchmark that spans concepts in justice, well-being, duties, virtues, and commonsense morality.
We find that current language models have a promising but incomplete ability to predict basic human ethical judgements.
Our work shows that progress can be made on machine ethics today, and it provides a steppingstone toward AI that is aligned with human values.
arXiv Detail & Related papers (2020-08-05T17:59:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.