Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmas

• URL: http://arxiv.org/abs/2305.00768v1
• Date: Mon, 1 May 2023 11:09:23 GMT
• Title: Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmas
• Authors: Udari Madhushani, Kevin R. McKee, John P. Agapiou, Joel Z. Leibo, Richard Everett, Thomas Anthony, Edward Hughes, Karl Tuyls, Edgar A. Du\'e\~nez-Guzm\'an
• Abstract summary: Social Value Orientation (SVO) describes an individual's propensity to allocate resources between themself and others. Prior studies show that groups of agents endowed with heterogeneous SVO learn diverse policies in settings that resemble the incentive structure of Prisoner's dilemma. We show that these best-response agents learn policies that are conditioned on their co-players, which we posit is the reason for improved zero-shot generalization results.
• Score: 15.171556039829161
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In social psychology, Social Value Orientation (SVO) describes an individual's propensity to allocate resources between themself and others. In reinforcement learning, SVO has been instantiated as an intrinsic motivation that remaps an agent's rewards based on particular target distributions of group reward. Prior studies show that groups of agents endowed with heterogeneous SVO learn diverse policies in settings that resemble the incentive structure of Prisoner's dilemma. Our work extends this body of results and demonstrates that (1) heterogeneous SVO leads to meaningfully diverse policies across a range of incentive structures in sequential social dilemmas, as measured by task-specific diversity metrics; and (2) learning a best response to such policy diversity leads to better zero-shot generalization in some situations. We show that these best-response agents learn policies that are conditioned on their co-players, which we posit is the reason for improved zero-shot generalization results.

Related papers

• The State-Action-Reward-State-Action Algorithm in Spatial Prisoner's Dilemma Game [0.0]
  Reinforcement learning provides a suitable framework for studying evolutionary game theory. We employ the State-Action-Reward-State-Action algorithm as the decision-making mechanism for individuals in evolutionary game theory. We evaluate the impact of SARSA on cooperation rates by analyzing variations in rewards and the distribution of cooperators and defectors within the network.
  arXiv  Detail & Related papers  (2024-06-25T07:21:35Z)
• Agent Alignment in Evolving Social Norms [65.45423591744434]
  We propose an evolutionary framework for agent evolution and alignment, named EvolutionaryAgent. In an environment where social norms continuously evolve, agents better adapted to the current social norms will have a higher probability of survival and proliferation. We show that EvolutionaryAgent can align progressively better with the evolving social norms while maintaining its proficiency in general tasks.
  arXiv  Detail & Related papers  (2024-01-09T15:44:44Z)
• Social Value Orientation and Integral Emotions in Multi-Agent Systems [1.5469452301122173]
  Human social behavior is influenced by individual differences in social preferences. Social value orientation (SVO) is a measurable personality trait. Integral emotions, the emotions which arise in direct response to a decision-making scenario, have been linked to temporary shifts in decision-making preferences.
  arXiv  Detail & Related papers  (2023-05-09T15:33:50Z)
• Stable Bias: Analyzing Societal Representations in Diffusion Models [72.27121528451528]
  We propose a new method for exploring the social biases in Text-to-Image (TTI) systems. Our approach relies on characterizing the variation in generated images triggered by enumerating gender and ethnicity markers in the prompts. We leverage this method to analyze images generated by 3 popular TTI systems and find that while all of their outputs show correlations with US labor demographics, they also consistently under-represent marginalized identities to different extents.
  arXiv  Detail & Related papers  (2023-03-20T19:32:49Z)
• Learning Roles with Emergent Social Value Orientations [49.16026283952117]
  This paper introduces the typical "division of labor or roles" mechanism in human society. We provide a promising solution for intertemporal social dilemmas (ISD) with social value orientations (SVO) A novel learning framework, called Learning Roles with Emergent SVOs (RESVO), is proposed to transform the learning of roles into the social value orientation emergence.
  arXiv  Detail & Related papers  (2023-01-31T17:54:09Z)
• Social Diversity Reduces the Complexity and Cost of Fostering Fairness [63.70639083665108]
  We investigate the effects of interference mechanisms which assume incomplete information and flexible standards of fairness. We quantify the role of diversity and show how it reduces the need for information gathering. Our results indicate that diversity changes and opens up novel mechanisms available to institutions wishing to promote fairness.
  arXiv  Detail & Related papers  (2022-11-18T21:58:35Z)
• Learning Collective Action under Risk Diversity [68.88688248278102]
  We investigate the consequences of risk diversity in groups of agents learning to play collective risk dilemmas. We show that risk diversity significantly reduces overall cooperation and hinders collective target achievement. Our results highlight the need for aligning risk perceptions among agents or develop new learning techniques.
  arXiv  Detail & Related papers  (2022-01-30T18:21:21Z)
• PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings [69.87899694963251]
  We propose a novel Peer-Contrastive Learning (PCL) with diverse augmentations. PCL constructs diverse contrastive positives and negatives at the group level for unsupervised sentence embeddings. PCL can perform peer-positive contrast as well as peer-network cooperation, which offers an inherent anti-bias ability.
  arXiv  Detail & Related papers  (2022-01-28T13:02:41Z)
• Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks [2.541277269153809]
  We propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation. We show that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma. We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies.
  arXiv  Detail & Related papers  (2021-10-19T08:40:56Z)
• Social diversity and social preferences in mixed-motive reinforcement learning [11.010593309447067]
  Studies of reinforcement learning in mixed-motive games have primarily leveraged homogeneous approaches. We study the effect of population heterogeneity on mixed-motive reinforcement learning.
  arXiv  Detail & Related papers  (2020-02-06T16:07:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.