Counterfactual Effect Decomposition in Multi-Agent Sequential Decision Making

• URL: http://arxiv.org/abs/2410.12539v1
• Date: Wed, 16 Oct 2024 13:20:35 GMT
• Title: Counterfactual Effect Decomposition in Multi-Agent Sequential Decision Making
• Authors: Stelios Triantafyllou, Aleksa Sukovic, Yasaman Zolfimoselo, Goran Radanovic,
• Abstract summary: We introduce a novel causal explanation formula that decomposes the counterfactual effect by attributing to each agent and state variable a score reflecting their respective contributions to the effect. We show that the total counterfactual effect of an agent's action can be decomposed into two components: one measuring the effect that propagates through all subsequent agents' actions and another related to the effect that propagates through the state transitions.
• Score: 9.469649321687928
• License:
• Abstract: We address the challenge of explaining counterfactual outcomes in multi-agent Markov decision processes. In particular, we aim to explain the total counterfactual effect of an agent's action on the outcome of a realized scenario through its influence on the environment dynamics and the agents' behavior. To achieve this, we introduce a novel causal explanation formula that decomposes the counterfactual effect by attributing to each agent and state variable a score reflecting their respective contributions to the effect. First, we show that the total counterfactual effect of an agent's action can be decomposed into two components: one measuring the effect that propagates through all subsequent agents' actions and another related to the effect that propagates through the state transitions. Building on recent advancements in causal contribution analysis, we further decompose these two effects as follows. For the former, we consider agent-specific effects -- a causal concept that quantifies the counterfactual effect of an agent's action that propagates through a subset of agents. Based on this notion, we use Shapley value to attribute the effect to individual agents. For the latter, we consider the concept of structure-preserving interventions and attribute the effect to state variables based on their "intrinsic" contributions. Through extensive experimentation, we demonstrate the interpretability of our decomposition approach in a Gridworld environment with LLM-assisted agents and a sepsis management simulator.

Related papers

• Causal Influence in Federated Edge Inference [34.487472866247586]
  In this paper, we consider a setting where heterogeneous agents with connectivity are performing inference using unlabeled streaming data. In order to overcome the uncertainty, agents cooperate with each other by exchanging their local inferences with and through a fusion center. Various scenarios reflecting different agent participation patterns and fusion center policies are investigated.
  arXiv  Detail & Related papers  (2024-05-02T13:06:50Z)
• Neural Interaction Energy for Multi-Agent Trajectory Prediction [55.098754835213995]
  We introduce a framework called Multi-Agent Trajectory prediction via neural interaction Energy (MATE) MATE assesses the interactive motion of agents by employing neural interaction energy. To bolster temporal stability, we introduce two constraints: inter-agent interaction constraint and intra-agent motion constraint.
  arXiv  Detail & Related papers  (2024-04-25T12:47:47Z)
• Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPs [13.524274041966539]
  We introduce agent-specific effects (ASE), a novel causal quantity that measures the effect of an agent's action on the outcome that propagates through other agents. We experimentally evaluate the utility of cf-ASE through a simulation-based testbed, which includes a sepsis management environment.
  arXiv  Detail & Related papers  (2023-10-17T15:12:56Z)
• A Causal Framework for Decomposing Spurious Variations [68.12191782657437]
  We develop tools for decomposing spurious variations in Markovian and Semi-Markovian models. We prove the first results that allow a non-parametric decomposition of spurious effects. The described approach has several applications, ranging from explainable and fair AI to questions in epidemiology and medicine.
  arXiv  Detail & Related papers  (2023-06-08T09:40:28Z)
• Inferring Individual Direct Causal Effects Under Heterogeneous Peer Influence [10.609670658904562]
  Causal inference in networks should account for interference, which occurs when a unit's outcome is influenced by treatments or outcomes of peers. We propose a structural causal model for networks that can capture different possible assumptions about network structure, interference conditions, and causal dependence. We find potential heterogeneous contexts using the causal model and propose a novel graph neural network-based estimator to estimate individual direct causal effects.
  arXiv  Detail & Related papers  (2023-05-27T13:57:26Z)
• Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics [57.671493865825255]
  We propose to model the impact of actions on-the-fly using latent embeddings. By combining these latent action embeddings with a novel, transformer-based, policy head, we design an Action Adaptive Policy. We show that our AAP is highly performant even when faced, at inference-time with missing actions and, previously unseen, perturbed action space.
  arXiv  Detail & Related papers  (2023-04-24T17:35:47Z)
• A Reinforcement Learning Framework for Dynamic Mediation Analysis [16.284199152492487]
  We propose a reinforcement learning framework to evaluate dynamic mediation effects in settings with infinite horizons. We decompose the average treatment effect into an immediate direct effect, an immediate mediation effect, a delayed direct effect, and a delayed mediation effect. We develop robust and semi-parametrically efficient estimators under the RL framework to infer these causal effects.
  arXiv  Detail & Related papers  (2023-01-31T00:50:05Z)
• Mitigating Negative Side Effects via Environment Shaping [27.400267388362654]
  Agents operating in unstructured environments often produce negative side effects (NSE) We present an algorithm to solve this problem and analyze its theoretical properties. Empirical evaluation of our approach shows that the proposed framework can successfully mitigate NSE, without affecting the agent's ability to complete its assigned task.
  arXiv  Detail & Related papers  (2021-02-13T22:15:00Z)
• Loss Bounds for Approximate Influence-Based Abstraction [81.13024471616417]
  Influence-based abstraction aims to gain leverage by modeling local subproblems together with the 'influence' that the rest of the system exerts on them. This paper investigates the performance of such approaches from a theoretical perspective. We show that neural networks trained with cross entropy are well suited to learn approximate influence representations.
  arXiv  Detail & Related papers  (2020-11-03T15:33:10Z)
• Understanding Adversarial Examples from the Mutual Influence of Images and Perturbations [83.60161052867534]
  We analyze adversarial examples by disentangling the clean images and adversarial perturbations, and analyze their influence on each other. Our results suggest a new perspective towards the relationship between images and universal perturbations. We are the first to achieve the challenging task of a targeted universal attack without utilizing original training data.
  arXiv  Detail & Related papers  (2020-07-13T05:00:09Z)
• Self-Attention Attribution: Interpreting Information Interactions Inside Transformer [89.21584915290319]
  We propose a self-attention attribution method to interpret the information interactions inside Transformer. We show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.
  arXiv  Detail & Related papers  (2020-04-23T14:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.