DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting

• URL: http://arxiv.org/abs/2403.02914v1
• Date: Tue, 5 Mar 2024 12:31:24 GMT
• Title: DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting
• Authors: Hao Wu, Haomin Wen, Guibin Zhang, Yutong Xia, Kai Wang, Yuxuan Liang, Yu Zheng, Kun Wang
• Abstract summary: Earth science systems rely heavily on the extensive deployment of sensors. Traditional approaches to sensor deployment utilize specific algorithms to design and deploy sensors. In this paper, we introduce for the first time the concept of dynamic sparse training and are committed to adaptively, dynamically filtering important sensor data.
• Score: 24.00162014044092
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ever-increasing sensor service, though opening a precious path and providing a deluge of earth system data for deep-learning-oriented earth science, sadly introduce a daunting obstacle to their industrial level deployment. Concretely, earth science systems rely heavily on the extensive deployment of sensors, however, the data collection from sensors is constrained by complex geographical and social factors, making it challenging to achieve comprehensive coverage and uniform deployment. To alleviate the obstacle, traditional approaches to sensor deployment utilize specific algorithms to design and deploy sensors. These methods dynamically adjust the activation times of sensors to optimize the detection process across each sub-region. Regrettably, formulating an activation strategy generally based on historical observations and geographic characteristics, which make the methods and resultant models were neither simple nor practical. Worse still, the complex technical design may ultimately lead to a model with weak generalizability. In this paper, we introduce for the first time the concept of spatio-temporal data dynamic sparse training and are committed to adaptively, dynamically filtering important sensor distributions. To our knowledge, this is the first proposal (termed DynST) of an industry-level deployment optimization concept at the data level. However, due to the existence of the temporal dimension, pruning of spatio-temporal data may lead to conflicts at different timestamps. To achieve this goal, we employ dynamic merge technology, along with ingenious dimensional mapping to mitigate potential impacts caused by the temporal aspect. During the training process, DynST utilize iterative pruning and sparse training, repeatedly identifying and dynamically removing sensor perception areas that contribute the least to future predictions.

Related papers

• Conservation-informed Graph Learning for Spatiotemporal Dynamics Prediction [84.26340606752763]
  In this paper, we introduce the conservation-informed GNN (CiGNN), an end-to-end explainable learning framework. The network is designed to conform to the general symmetry conservation law via symmetry where conservative and non-conservative information passes over a multiscale space by a latent temporal marching strategy. Results demonstrate that CiGNN exhibits remarkable baseline accuracy and generalizability, and is readily applicable to learning for prediction of varioustemporal dynamics.
  arXiv  Detail & Related papers  (2024-12-30T13:55:59Z)
• Kriformer: A Novel Spatiotemporal Kriging Approach Based on Graph Transformers [5.4381914710364665]
  This study addresses posed by sparse sensor deployment and unreliable data by framing the problem as an environmental challenge. A graphkriformer model, Kriformer, estimates data at locations without sensors by mining spatial and temporal correlations, even with limited resources.
  arXiv  Detail & Related papers  (2024-09-23T11:01:18Z)
• Rethinking Sensors Modeling: Hierarchical Information Enhanced Traffic Forecasting [47.1051445072085]
  We argue to rethink the sensor's dependency modeling from two hierarchies: regional and global. We generate representative and common-temporal patterns as global nodes to reflect a global dependency between sensors. In pursuit of the generality of reality of node representations, we incorporate a Meta GCN to propagate and global nodes in the physical data space.
  arXiv  Detail & Related papers  (2023-09-20T13:08:34Z)
• Spatio-Temporal Branching for Motion Prediction using Motion Increments [55.68088298632865]
  Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques. We propose a noveltemporal-temporal branching network using incremental information for HMP.
  arXiv  Detail & Related papers  (2023-08-02T12:04:28Z)
• Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstruction [4.243926243206826]
  We show that a sequence-to-vector model, such as an LSTM (long, short-term memory) network, with a decoder network, dynamic information can be mapped to full state-space estimates. The exceptional performance of the network architecture is demonstrated on three applications.
  arXiv  Detail & Related papers  (2023-07-20T21:42:01Z)
• RIOT: Recursive Inertial Odometry Transformer for Localisation from Low-Cost IMU Measurements [5.770538064283154]
  We present two end-to-end frameworks for pose invariant deep inertial odometry that utilise self-attention to capture both spatial features and long-range dependencies in inertial data. We evaluate our approaches against a custom 2-layer Gated Recurrent Unit, trained in the same manner on the same data, and tested each approach on a number of different users, devices and activities.
  arXiv  Detail & Related papers  (2023-03-03T00:20:01Z)
• Temporal Predictive Coding For Model-Based Planning In Latent Space [80.99554006174093]
  We present an information-theoretic approach that employs temporal predictive coding to encode elements in the environment that can be predicted across time. We evaluate our model on a challenging modification of standard DMControl tasks where the background is replaced with natural videos that contain complex but irrelevant information to the planning task.
  arXiv  Detail & Related papers  (2021-06-14T04:31:15Z)
• Adaptive Latent Space Tuning for Non-Stationary Distributions [62.997667081978825]
  We present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs. We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator.
  arXiv  Detail & Related papers  (2021-05-08T03:50:45Z)
• Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning [0.0]
  We propose a data-driven spatial field recovery technique based on a structured grid-based deep-learning approach for arbitrary positioned sensors of any numbers. The presented technique opens a new pathway towards the practical use of neural networks for real-time global field estimation.
  arXiv  Detail & Related papers  (2021-01-03T03:43:53Z)
• Deep Learning Framework for Detecting Ground Deformation in the Built Environment using Satellite InSAR data [7.503635457124339]
  We adapt a pre-trained convolutional neural network (CNN) to detect deformation in a national-scale velocity field. We focus on the UK where previously identified deformation is associated with coal-mining, ground water withdrawal, landslides and tunnelling. The results demonstrate the potential applicability of the proposed framework to the development of automated ground motion analysis systems.
  arXiv  Detail & Related papers  (2020-05-07T03:14:00Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.