Amortized Network Intervention to Steer the Excitatory Point Processes
- URL: http://arxiv.org/abs/2310.04159v2
- Date: Mon, 15 Apr 2024 12:52:30 GMT
- Title: Amortized Network Intervention to Steer the Excitatory Point Processes
- Authors: Zitao Song, Wendi Ren, Shuang Li,
- Abstract summary: Excitatory point processes (i.e., event flows) occurring over dynamic graphs provide a fine-grained model to capture how discrete events may spread over time and space.
How to effectively steer the event flows by modifying the dynamic graph structures presents an interesting problem, motivated by curbing the spread of infectious diseases.
We design an Amortized Network Interventions framework, allowing for the pooling of optimal policies from history and other contexts.
- Score: 8.15558505134853
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Excitatory point processes (i.e., event flows) occurring over dynamic graphs (i.e., evolving topologies) provide a fine-grained model to capture how discrete events may spread over time and space. How to effectively steer the event flows by modifying the dynamic graph structures presents an interesting problem, motivated by curbing the spread of infectious diseases through strategically locking down cities to mitigating traffic congestion via traffic light optimization. To address the intricacies of planning and overcome the high dimensionality inherent to such decision-making problems, we design an Amortized Network Interventions (ANI) framework, allowing for the pooling of optimal policies from history and other contexts while ensuring a permutation equivalent property. This property enables efficient knowledge transfer and sharing across diverse contexts. Each task is solved by an H-step lookahead model-based reinforcement learning, where neural ODEs are introduced to model the dynamics of the excitatory point processes. Instead of simulating rollouts from the dynamics model, we derive an analytical mean-field approximation for the event flows given the dynamics, making the online planning more efficiently solvable. We empirically illustrate that this ANI approach substantially enhances policy learning for unseen dynamics and exhibits promising outcomes in steering event flows through network intervention using synthetic and real COVID datasets.
Related papers
- Energy-Guided Data Sampling for Traffic Prediction with Mini Training Datasets [13.065729535009925]
We propose an innovative solution that merges Convolutional Neural Networks (CNNs) with Long Short-Term Memory (LSTM) architecture to enhance the prediction of traffic flow dynamics.
A key revelation of our research is the feasibility of sampling training data for large traffic systems from simulations conducted on smaller traffic systems.
arXiv Detail & Related papers (2024-03-27T15:57:42Z) - Online Boosting Adaptive Learning under Concept Drift for Multistream
Classification [34.64751041290346]
Multistream classification poses significant challenges due to the necessity for rapid adaptation in dynamic streaming processes with concept drift.
We propose a novel Online Boosting Adaptive Learning (OBAL) method that adaptively learns the dynamic correlation among different streams.
arXiv Detail & Related papers (2023-12-17T23:10:39Z) - Latent Variable Representation for Reinforcement Learning [131.03944557979725]
It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning.
We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle.
In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
arXiv Detail & Related papers (2022-12-17T00:26:31Z) - GINK: Graph-based Interaction-aware Kinodynamic Planning via
Reinforcement Learning for Autonomous Driving [10.782043595405831]
There are many challenges in applying deep reinforcement learning (D) to autonomous driving in a structured environment such as an urban area.
In this paper, we suggest a new framework that effectively combines graph-based intention representation and reinforcement learning for dynamic planning.
The experiments show the state-of-the-art performance of our approach compared to the existing baselines.
arXiv Detail & Related papers (2022-06-03T10:37:25Z) - Data-Driven Traffic Assignment: A Novel Approach for Learning Traffic
Flow Patterns Using a Graph Convolutional Neural Network [1.3706331473063877]
We present a novel data-driven approach of learning traffic flow patterns of a transportation network.
We develop a neural network-based framework known as Graph Convolutional Neural Network (GCNN) to solve it.
When the training of the model is complete, it can instantly determine the traffic flows of a large-scale network.
arXiv Detail & Related papers (2022-02-21T19:45:15Z) - Learning Neural Causal Models with Active Interventions [83.44636110899742]
We introduce an active intervention-targeting mechanism which enables a quick identification of the underlying causal structure of the data-generating process.
Our method significantly reduces the required number of interactions compared with random intervention targeting.
We demonstrate superior performance on multiple benchmarks from simulated to real-world data.
arXiv Detail & Related papers (2021-09-06T13:10:37Z) - A purely data-driven framework for prediction, optimization, and control
of networked processes: application to networked SIS epidemic model [0.8287206589886881]
We develop a data-driven framework based on operator-theoretic techniques to identify and control nonlinear dynamics over large-scale networks.
The proposed approach requires no prior knowledge of the network structure and identifies the underlying dynamics solely using a collection of two-step snapshots of the states.
arXiv Detail & Related papers (2021-08-01T03:57:10Z) - Learning to Continuously Optimize Wireless Resource in a Dynamic
Environment: A Bilevel Optimization Perspective [52.497514255040514]
This work develops a new approach that enables data-driven methods to continuously learn and optimize resource allocation strategies in a dynamic environment.
We propose to build the notion of continual learning into wireless system design, so that the learning model can incrementally adapt to the new episodes.
Our design is based on a novel bilevel optimization formulation which ensures certain fairness" across different data samples.
arXiv Detail & Related papers (2021-05-03T07:23:39Z) - An Ode to an ODE [78.97367880223254]
We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d)
This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
arXiv Detail & Related papers (2020-06-19T22:05:19Z) - Network Diffusions via Neural Mean-Field Dynamics [52.091487866968286]
We propose a novel learning framework for inference and estimation problems of diffusion on networks.
Our framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node infection probabilities.
Our approach is versatile and robust to variations of the underlying diffusion network models.
arXiv Detail & Related papers (2020-06-16T18:45:20Z) - Context-aware Dynamics Model for Generalization in Model-Based
Reinforcement Learning [124.9856253431878]
We decompose the task of learning a global dynamics model into two stages: (a) learning a context latent vector that captures the local dynamics, then (b) predicting the next state conditioned on it.
In order to encode dynamics-specific information into the context latent vector, we introduce a novel loss function that encourages the context latent vector to be useful for predicting both forward and backward dynamics.
The proposed method achieves superior generalization ability across various simulated robotics and control tasks, compared to existing RL schemes.
arXiv Detail & Related papers (2020-05-14T08:10:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.