DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting

• URL: http://arxiv.org/abs/2403.02914v1
• Date: Tue, 5 Mar 2024 12:31:24 GMT
• Title: DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecasting
• Authors: Hao Wu, Haomin Wen, Guibin Zhang, Yutong Xia, Kai Wang, Yuxuan Liang, Yu Zheng, Kun Wang
• Abstract summary: Earth science systems rely heavily on the extensive deployment of sensors. Traditional approaches to sensor deployment utilize specific algorithms to design and deploy sensors. In this paper, we introduce for the first time the concept of dynamic sparse training and are committed to adaptively, dynamically filtering important sensor data.
• Score: 24.00162014044092
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ever-increasing sensor service, though opening a precious path and providing a deluge of earth system data for deep-learning-oriented earth science, sadly introduce a daunting obstacle to their industrial level deployment. Concretely, earth science systems rely heavily on the extensive deployment of sensors, however, the data collection from sensors is constrained by complex geographical and social factors, making it challenging to achieve comprehensive coverage and uniform deployment. To alleviate the obstacle, traditional approaches to sensor deployment utilize specific algorithms to design and deploy sensors. These methods dynamically adjust the activation times of sensors to optimize the detection process across each sub-region. Regrettably, formulating an activation strategy generally based on historical observations and geographic characteristics, which make the methods and resultant models were neither simple nor practical. Worse still, the complex technical design may ultimately lead to a model with weak generalizability. In this paper, we introduce for the first time the concept of spatio-temporal data dynamic sparse training and are committed to adaptively, dynamically filtering important sensor distributions. To our knowledge, this is the first proposal (termed DynST) of an industry-level deployment optimization concept at the data level. However, due to the existence of the temporal dimension, pruning of spatio-temporal data may lead to conflicts at different timestamps. To achieve this goal, we employ dynamic merge technology, along with ingenious dimensional mapping to mitigate potential impacts caused by the temporal aspect. During the training process, DynST utilize iterative pruning and sparse training, repeatedly identifying and dynamically removing sensor perception areas that contribute the least to future predictions.

Related papers

• Physics-Enhanced Graph Neural Networks For Soft Sensing in Industrial Internet of Things [6.374763930914524]
  The Industrial Internet of Things (IIoT) is reshaping manufacturing, industrial processes, and infrastructure management. achieving highly reliable IIoT can be hindered by factors such as the cost of installing large numbers of sensors, limitations in retrofitting existing systems with sensors, or harsh environmental conditions that may make sensor installation impractical. We propose physics-enhanced Graph Neural Networks (GNNs), which integrate principles of physics into graph-based methodologies.
  arXiv  Detail & Related papers  (2024-04-11T18:03:59Z)
• Rethinking Sensors Modeling: Hierarchical Information Enhanced Traffic Forecasting [47.1051445072085]
  We argue to rethink the sensor's dependency modeling from two hierarchies: regional and global. We generate representative and common-temporal patterns as global nodes to reflect a global dependency between sensors. In pursuit of the generality of reality of node representations, we incorporate a Meta GCN to propagate and global nodes in the physical data space.
  arXiv  Detail & Related papers  (2023-09-20T13:08:34Z)
• Spatio-Temporal Branching for Motion Prediction using Motion Increments [55.68088298632865]
  Human motion prediction (HMP) has emerged as a popular research topic due to its diverse applications. Traditional methods rely on hand-crafted features and machine learning techniques. We propose a noveltemporal-temporal branching network using incremental information for HMP.
  arXiv  Detail & Related papers  (2023-08-02T12:04:28Z)
• Leveraging arbitrary mobile sensor trajectories with shallow recurrent decoder networks for full-state reconstruction [4.243926243206826]
  We show that a sequence-to-vector model, such as an LSTM (long, short-term memory) network, with a decoder network, dynamic information can be mapped to full state-space estimates. The exceptional performance of the network architecture is demonstrated on three applications.
  arXiv  Detail & Related papers  (2023-07-20T21:42:01Z)
• RIOT: Recursive Inertial Odometry Transformer for Localisation from Low-Cost IMU Measurements [5.770538064283154]
  We present two end-to-end frameworks for pose invariant deep inertial odometry that utilise self-attention to capture both spatial features and long-range dependencies in inertial data. We evaluate our approaches against a custom 2-layer Gated Recurrent Unit, trained in the same manner on the same data, and tested each approach on a number of different users, devices and activities.
  arXiv  Detail & Related papers  (2023-03-03T00:20:01Z)
• Adaptive Latent Space Tuning for Non-Stationary Distributions [62.997667081978825]
  We present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs. We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator.
  arXiv  Detail & Related papers  (2021-05-08T03:50:45Z)
• Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data [64.40187171234838]
  Seasonal Contrast (SeCo) is an effective pipeline to leverage unlabeled data for in-domain pre-training of re-mote sensing representations. SeCo will be made public to facilitate transfer learning and enable rapid progress in re-mote sensing applications.
  arXiv  Detail & Related papers  (2021-03-30T18:26:39Z)
• Global field reconstruction from sparse sensors with Voronoi tessellation-assisted deep learning [0.0]
  We propose a data-driven spatial field recovery technique based on a structured grid-based deep-learning approach for arbitrary positioned sensors of any numbers. The presented technique opens a new pathway towards the practical use of neural networks for real-time global field estimation.
  arXiv  Detail & Related papers  (2021-01-03T03:43:53Z)
• Energy Aware Deep Reinforcement Learning Scheduling for Sensors Correlated in Time and Space [62.39318039798564]
  We propose a scheduling mechanism capable of taking advantage of correlated information. The proposed mechanism is capable of determining the frequency with which sensors should transmit their updates. We show that our solution can significantly extend the sensors' lifetime.
  arXiv  Detail & Related papers  (2020-11-19T09:53:27Z)
• Data-Driven Distributed State Estimation and Behavior Modeling in Sensor Networks [5.817715558396024]
  We formulate the problem of simultaneous state estimation and behavior learning in a sensor network. We propose a simple yet effective solution by extending the Gaussian process-based Bayes filters (GP-BayesFilters) to an online, distributed setting. The effectiveness of the proposed method is evaluated on tracking objects with unknown movement behaviors using both synthetic data and data collected from a multi-robot platform.
  arXiv  Detail & Related papers  (2020-09-22T21:31:18Z)
• Deep Soft Procrustes for Markerless Volumetric Sensor Alignment [81.13055566952221]
  In this work, we improve markerless data-driven correspondence estimation to achieve more robust multi-sensor spatial alignment. We incorporate geometric constraints in an end-to-end manner into a typical segmentation based model and bridge the intermediate dense classification task with the targeted pose estimation one. Our model is experimentally shown to achieve similar results with marker-based methods and outperform the markerless ones, while also being robust to the pose variations of the calibration structure.
  arXiv  Detail & Related papers  (2020-03-23T10:51:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.