PIMbot: Policy and Incentive Manipulation for Multi-Robot Reinforcement Learning in Social Dilemmas

• URL: http://arxiv.org/abs/2307.15944v1
• Date: Sat, 29 Jul 2023 09:34:45 GMT
• Title: PIMbot: Policy and Incentive Manipulation for Multi-Robot Reinforcement Learning in Social Dilemmas
• Authors: Shahab Nikkhoo, Zexin Li, Aritra Samanta, Yufei Li and Cong Liu
• Abstract summary: This paper presents a novel approach, namely PIMbot, to manipulating the reward function in multi-robot collaboration. By utilizing our proposed PIMbot mechanisms, a robot is able to manipulate the social dilemma environment effectively. Our work provides insights into how inter-robot communication can be manipulated and has implications for various robotic applications.
• Score: 4.566617428324801
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent research has demonstrated the potential of reinforcement learning (RL) in enabling effective multi-robot collaboration, particularly in social dilemmas where robots face a trade-off between self-interests and collective benefits. However, environmental factors such as miscommunication and adversarial robots can impact cooperation, making it crucial to explore how multi-robot communication can be manipulated to achieve different outcomes. This paper presents a novel approach, namely PIMbot, to manipulating the reward function in multi-robot collaboration through two distinct forms of manipulation: policy and incentive manipulation. Our work introduces a new angle for manipulation in recent multi-agent RL social dilemmas that utilize a unique reward function for incentivization. By utilizing our proposed PIMbot mechanisms, a robot is able to manipulate the social dilemma environment effectively. PIMbot has the potential for both positive and negative impacts on the task outcome, where positive impacts lead to faster convergence to the global optimum and maximized rewards for any chosen robot. Conversely, negative impacts can have a detrimental effect on the overall task performance. We present comprehensive experimental results that demonstrate the effectiveness of our proposed methods in the Gazebo-simulated multi-robot environment. Our work provides insights into how inter-robot communication can be manipulated and has implications for various robotic applications. %, including robotics, transportation, and manufacturing.

Related papers

• Human-Robot Mutual Learning through Affective-Linguistic Interaction and Differential Outcomes Training [Pre-Print] [0.3811184252495269]
  We test how affective-linguistic communication, in combination with differential outcomes training, affects mutual learning in a human-robot context. Taking inspiration from child- caregiver dynamics, our human-robot interaction setup consists of a (simulated) robot attempting to learn how best to communicate internal, homeostatically-controlled needs.
  arXiv  Detail & Related papers  (2024-07-01T13:35:08Z)
• Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning [54.636562516974884]
  In imitation and reinforcement learning, the cost of human supervision limits the amount of data that robots can be trained on. In this work, we propose MEDAL++, a novel design for self-improving robotic systems. The robot autonomously practices the task by learning to both do and undo the task, simultaneously inferring the reward function from the demonstrations.
  arXiv  Detail & Related papers  (2023-03-02T18:51:38Z)
• REvolveR: Continuous Evolutionary Models for Robot-to-robot Policy Transfer [57.045140028275036]
  We consider the problem of transferring a policy across two different robots with significantly different parameters such as kinematics and morphology. Existing approaches that train a new policy by matching the action or state transition distribution, including imitation learning methods, fail due to optimal action and/or state distribution being mismatched in different robots. We propose a novel method named $REvolveR$ of using continuous evolutionary models for robotic policy transfer implemented in a physics simulator.
  arXiv  Detail & Related papers  (2022-02-10T18:50:25Z)
• Doing Right by Not Doing Wrong in Human-Robot Collaboration [8.078753289996417]
  We propose a novel approach to learning fair and sociable behavior, not by reproducing positive behavior, but rather by avoiding negative behavior. In this study, we highlight the importance of incorporating sociability in robot manipulation, as well as the need to consider fairness in human-robot interactions.
  arXiv  Detail & Related papers  (2022-02-05T23:05:10Z)
• Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together [68.44697646919515]
  This paper presents several human-robot systems that utilize spatial computing to enable novel robot use cases. The combination of spatial computing and egocentric sensing on mixed reality devices enables them to capture and understand human actions and translate these to actions with spatial meaning.
  arXiv  Detail & Related papers  (2022-02-03T10:04:26Z)
• A MultiModal Social Robot Toward Personalized Emotion Interaction [1.2183405753834562]
  This study demonstrates a multimodal human-robot interaction (HRI) framework with reinforcement learning to enhance the robotic interaction policy. The goal is to apply this framework in social scenarios that can let the robots generate a more natural and engaging HRI framework.
  arXiv  Detail & Related papers  (2021-10-08T00:35:44Z)
• Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
  Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times. In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems. We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
  arXiv  Detail & Related papers  (2021-09-19T18:00:51Z)
• Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration [51.268988527778276]
  We present a method for learning a human-robot collaboration policy from human-human collaboration demonstrations. Our method co-optimizes a human policy and a robot policy in an interactive learning process.
  arXiv  Detail & Related papers  (2021-08-13T03:14:43Z)
• Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration [83.4081612443128]
  We show that a short calibration using REMP can effectively bridge the gap between what a non-expert user thinks a robot can reach and the ground-truth. We show that this calibration procedure not only results in better user perception, but also promotes more efficient human-robot collaborations.
  arXiv  Detail & Related papers  (2021-03-06T09:14:30Z)
• Supportive Actions for Manipulation in Human-Robot Coworker Teams [15.978389978586414]
  We term actions that support interaction by reducing future interference with others as supportive robot actions. We compare two robot modes in a shared table pick-and-place task: (1) Task-oriented: the robot only takes actions to further its own task objective and (2) Supportive: the robot sometimes prefers supportive actions to task-oriented ones. Our experiments in simulation, using a simplified human model, reveal that supportive actions reduce the interference between agents, especially in more difficult tasks, but also cause the robot to take longer to complete the task.
  arXiv  Detail & Related papers  (2020-05-02T09:37:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.