How Do You Act? An Empirical Study to Understand Behavior of Deep Reinforcement Learning Agents

• URL: http://arxiv.org/abs/2004.03237v1
• Date: Tue, 7 Apr 2020 10:08:55 GMT
• Title: How Do You Act? An Empirical Study to Understand Behavior of Deep Reinforcement Learning Agents
• Authors: Richard Meyes, Moritz Schneider, Tobias Meisen
• Abstract summary: The demand for more transparency of decision-making processes of deep reinforcement learning agents is greater than ever. In this study, we characterize the learned representations of an agent's policy network through its activation space. We show that the healthy agent's behavior is characterized by a distinct correlation pattern between the network's layer activation and the performed actions.
• Score: 2.3268634502937937
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The demand for more transparency of decision-making processes of deep reinforcement learning agents is greater than ever, due to their increased use in safety critical and ethically challenging domains such as autonomous driving. In this empirical study, we address this lack of transparency following an idea that is inspired by research in the field of neuroscience. We characterize the learned representations of an agent's policy network through its activation space and perform partial network ablations to compare the representations of the healthy and the intentionally damaged networks. We show that the healthy agent's behavior is characterized by a distinct correlation pattern between the network's layer activation and the performed actions during an episode and that network ablations, which cause a strong change of this pattern, lead to the agent failing its trained control task. Furthermore, the learned representation of the healthy agent is characterized by a distinct pattern in its activation space reflecting its different behavioral stages during an episode, which again, when distorted by network ablations, leads to the agent failing its trained control task. Concludingly, we argue in favor of a new perspective on artificial neural networks as objects of empirical investigations, just as biological neural systems in neuroscientific studies, paving the way towards a new standard of scientific falsifiability with respect to research on transparency and interpretability of artificial neural networks.

Related papers

• Evolving Neural Networks Reveal Emergent Collective Behavior from Minimal Agent Interactions [0.0]
  We investigate how neural networks evolve to control agents' behavior in a dynamic environment. Simpler behaviors, such as lane formation and laminar flow, are characterized by more linear network operations. Specific environmental parameters, such as moderate noise, broader field of view, and lower agent density, promote the evolution of non-linear networks.
  arXiv  Detail & Related papers  (2024-10-25T17:43:00Z)
• Identifying Sub-networks in Neural Networks via Functionally Similar Representations [41.028797971427124]
  We take a step toward automating the understanding of the network by investigating the existence of distinct sub-networks. Our approach offers meaningful insights into the behavior of neural networks with minimal human and computational cost.
  arXiv  Detail & Related papers  (2024-10-21T20:19:00Z)
• A Survey on Transferability of Adversarial Examples across Deep Neural Networks [53.04734042366312]
  adversarial examples can manipulate machine learning models into making erroneous predictions. The transferability of adversarial examples enables black-box attacks which circumvent the need for detailed knowledge of the target model. This survey explores the landscape of the adversarial transferability of adversarial examples.
  arXiv  Detail & Related papers  (2023-10-26T17:45:26Z)
• Investigating Human-Identifiable Features Hidden in Adversarial Perturbations [54.39726653562144]
  Our study explores up to five attack algorithms across three datasets. We identify human-identifiable features in adversarial perturbations. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models.
  arXiv  Detail & Related papers  (2023-09-28T22:31:29Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Abrupt and spontaneous strategy switches emerge in simple regularised neural networks [8.737068885923348]
  We study whether insight-like behaviour can occur in simple artificial neural networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism. This suggests that insight-like behaviour can arise naturally from gradual learning in simple neural networks.
  arXiv  Detail & Related papers  (2023-02-22T12:48:45Z)
• Searching for the Essence of Adversarial Perturbations [73.96215665913797]
  We show that adversarial perturbations contain human-recognizable information, which is the key conspirator responsible for a neural network's erroneous prediction. This concept of human-recognizable information allows us to explain key features related to adversarial perturbations.
  arXiv  Detail & Related papers  (2022-05-30T18:04:57Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Backprop-Free Reinforcement Learning with Active Neural Generative Coding [84.11376568625353]
  We propose a computational framework for learning action-driven generative models without backpropagation of errors (backprop) in dynamic environments. We develop an intelligent agent that operates even with sparse rewards, drawing inspiration from the cognitive theory of planning as inference. The robust performance of our agent offers promising evidence that a backprop-free approach for neural inference and learning can drive goal-directed behavior.
  arXiv  Detail & Related papers  (2021-07-10T19:02:27Z)
• Training spiking neural networks using reinforcement learning [0.0]
  We propose biologically-plausible alternatives to backpropagation to facilitate the training of spiking neural networks. We focus on investigating the candidacy of reinforcement learning rules in solving the spatial and temporal credit assignment problems. We compare and contrast the two approaches by applying them to traditional RL domains such as gridworld, cartpole and mountain car.
  arXiv  Detail & Related papers  (2020-05-12T17:40:36Z)
• Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations [0.3441021278275805]
  We shed light on the roles of single neurons and groups of neurons within the network fulfilling a learned task. We find that neither a neuron's magnitude or selectivity of activation, nor its impact on network performance are sufficient stand-alone indicators.
  arXiv  Detail & Related papers  (2020-04-02T20:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.