Counterfactual Data Augmentation using Locally Factored Dynamics

• URL: http://arxiv.org/abs/2007.02863v2
• Date: Thu, 3 Dec 2020 23:50:13 GMT
• Title: Counterfactual Data Augmentation using Locally Factored Dynamics
• Authors: Silviu Pitis, Elliot Creager, Animesh Garg
• Abstract summary: Local causal structures can be leveraged to improve the sample efficiency of sequence prediction and off-policy reinforcement learning. We propose an approach to inferring these structures given an object-oriented state representation, as well as a novel algorithm for Counterfactual Data Augmentation.
• Score: 44.37487079747397
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many dynamic processes, including common scenarios in robotic control and reinforcement learning (RL), involve a set of interacting subprocesses. Though the subprocesses are not independent, their interactions are often sparse, and the dynamics at any given time step can often be decomposed into locally independent causal mechanisms. Such local causal structures can be leveraged to improve the sample efficiency of sequence prediction and off-policy reinforcement learning. We formalize this by introducing local causal models (LCMs), which are induced from a global causal model by conditioning on a subset of the state space. We propose an approach to inferring these structures given an object-oriented state representation, as well as a novel algorithm for Counterfactual Data Augmentation (CoDA). CoDA uses local structures and an experience replay to generate counterfactual experiences that are causally valid in the global model. We find that CoDA significantly improves the performance of RL agents in locally factored tasks, including the batch-constrained and goal-conditioned settings.

Related papers

• STAT: Towards Generalizable Temporal Action Localization [56.634561073746056]
  Weakly-supervised temporal action localization (WTAL) aims to recognize and localize action instances with only video-level labels. Existing methods suffer from severe performance degradation when transferring to different distributions. We propose GTAL, which focuses on improving the generalizability of action localization methods.
  arXiv  Detail & Related papers  (2024-04-20T07:56:21Z)
• A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations [0.34410212782758043]
  Reinforcement Learning (RL) provides a powerful framework for decision-making in complex environments. This paper presents a novel Spiking Neural Network (SNN) architecture for solving RL problems with real-valued observations.
  arXiv  Detail & Related papers  (2023-07-06T12:33:34Z)
• ASR: Attention-alike Structural Re-parameterization [53.019657810468026]
  We propose a simple-yet-effective attention-alike structural re- parameterization (ASR) that allows us to achieve SRP for a given network while enjoying the effectiveness of the attention mechanism. In this paper, we conduct extensive experiments from a statistical perspective and discover an interesting phenomenon Stripe Observation, which reveals that channel attention values quickly approach some constant vectors during training.
  arXiv  Detail & Related papers  (2023-04-13T08:52:34Z)
• MoCoDA: Model-based Counterfactual Data Augmentation [40.878444530293635]
  We argue that the ability to recognize and use local factorization in transition dynamics is a key element in unlocking the power of multi-object reasoning. Knowing the local structure also allows us to predict which unseen states and actions this dynamics model will generalize to. We show that MoCoDA enables RL agents to learn policies that generalize to unseen states and actions.
  arXiv  Detail & Related papers  (2022-10-20T14:09:48Z)
• Change Detection for Local Explainability in Evolving Data Streams [72.4816340552763]
  Local feature attribution methods have become a popular technique for post-hoc and model-agnostic explanations. It is often unclear how local attributions behave in realistic, constantly evolving settings such as streaming and online applications. We present CDLEEDS, a flexible and model-agnostic framework for detecting local change and concept drift.
  arXiv  Detail & Related papers  (2022-09-06T18:38:34Z)
• Cross-modal Consensus Network for Weakly Supervised Temporal Action Localization [74.34699679568818]
  Weakly supervised temporal action localization (WS-TAL) is a challenging task that aims to localize action instances in the given video with video-level categorical supervision. We propose a cross-modal consensus network (CO2-Net) to tackle this problem.
  arXiv  Detail & Related papers  (2021-07-27T04:21:01Z)
• State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamics [0.0]
  We more systematically investigate the possibilty to use a local ensemble Kalman filter with either covariance localization or local domains. Global parameters are meant to represent the surrogate dynamics, while the local parameters typically stand for the forcings of the model. This paper more generally addresses the key question of online estimation of both global and local model parameters.
  arXiv  Detail & Related papers  (2021-07-23T14:12:20Z)
• Region-based Non-local Operation for Video Classification [11.746833714322154]
  This paper presents region-based non-local (RNL) operations as a family of self-attention mechanisms. By combining a channel attention module with the proposed RNL, we design an attention chain, which can be integrated into the off-the-shelf CNNs for end-to-end training. The experimental results of our method outperform other attention mechanisms, and we achieve state-of-the-art performance on the Something-Something V1 dataset.
  arXiv  Detail & Related papers  (2020-07-17T14:57:05Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)
• Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters [5.319361976450981]
  Federated learning is a distributed, privacy-aware learning scenario which trains a single model on data belonging to several clients. Current federated learning methods struggle in cases with heterogeneous client-side data distributions. We propose a novel representation matching scheme that reduces the divergence of local models.
  arXiv  Detail & Related papers  (2019-12-30T20:19:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.