Adaptive Latent Space Tuning for Non-Stationary Distributions
- URL: http://arxiv.org/abs/2105.03584v2
- Date: Wed, 12 May 2021 05:02:00 GMT
- Title: Adaptive Latent Space Tuning for Non-Stationary Distributions
- Authors: Alexander Scheinker, Frederick Cropp, Sergio Paiagua, Daniele
Filippetto
- Abstract summary: We present a method for adaptive tuning of the low-dimensional latent space of deep encoder-decoder style CNNs.
We demonstrate our approach for predicting the properties of a time-varying charged particle beam in a particle accelerator.
- Score: 62.997667081978825
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Powerful deep learning tools, such as convolutional neural networks (CNN),
are able to learn the input-output relationships of large complicated systems
directly from data. Encoder-decoder deep CNNs are able to extract features
directly from images, mix them with scalar inputs within a general
low-dimensional latent space, and then generate new complex 2D outputs which
represent complex physical phenomenon. One important challenge faced by deep
learning methods is large non-stationary systems whose characteristics change
quickly with time for which re-training is not feasible. In this paper we
present a method for adaptive tuning of the low-dimensional latent space of
deep encoder-decoder style CNNs based on real-time feedback to quickly
compensate for unknown and fast distribution shifts. We demonstrate our
approach for predicting the properties of a time-varying charged particle beam
in a particle accelerator whose components (accelerating electric fields and
focusing magnetic fields) are also quickly changing with time.
Related papers
- Navigating Spatio-Temporal Heterogeneity: A Graph Transformer Approach for Traffic Forecasting [13.309018047313801]
Traffic forecasting has emerged as a crucial research area in the development of smart cities.
Recent advancements in network modeling for most-temporal correlations are starting to see diminishing returns in performance.
To tackle these challenges, we introduce the Spatio-Temporal Graph Transformer (STGormer)
We design two straightforward yet effective spatial encoding methods based on the structure and integrate time position into the vanilla transformer to capture-temporal traffic patterns.
arXiv Detail & Related papers (2024-08-20T13:18:21Z) - Forecasting Fold Bifurcations through Physics-Informed Convolutional
Neural Networks [0.0]
This study proposes a physics-informed convolutional neural network (CNN) for identifying dynamical systems' time series near a fold bifurcation.
The CNN is trained with a relatively small amount of data and on a single, very simple system.
A similar task requires significant extrapolation capabilities, which are obtained by exploiting physics-based information.
arXiv Detail & Related papers (2023-12-21T10:07:52Z) - Disentangling Spatial and Temporal Learning for Efficient Image-to-Video
Transfer Learning [59.26623999209235]
We present DiST, which disentangles the learning of spatial and temporal aspects of videos.
The disentangled learning in DiST is highly efficient because it avoids the back-propagation of massive pre-trained parameters.
Extensive experiments on five benchmarks show that DiST delivers better performance than existing state-of-the-art methods by convincing gaps.
arXiv Detail & Related papers (2023-09-14T17:58:33Z) - A Comparison of Temporal Encoders for Neuromorphic Keyword Spotting with
Few Neurons [0.11726720776908518]
Two candidate neurocomputational elements for temporal encoding and feature extraction in SNNs are investigated.
Resource-efficient keyword spotting applications may benefit from the use of these encoders, but further work on methods for learning the time constants and weights is required.
arXiv Detail & Related papers (2023-01-24T12:50:54Z) - NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction.
The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network.
A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
arXiv Detail & Related papers (2022-09-29T04:06:00Z) - On Fast Simulation of Dynamical System with Neural Vector Enhanced
Numerical Solver [59.13397937903832]
We introduce a deep learning-based corrector called Neural Vector (NeurVec)
NeurVec can compensate for integration errors and enable larger time step sizes in simulations.
Our experiments on a variety of complex dynamical system benchmarks demonstrate that NeurVec exhibits remarkable generalization capability.
arXiv Detail & Related papers (2022-08-07T09:02:18Z) - Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs.
By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
arXiv Detail & Related papers (2022-02-01T17:11:13Z) - Adaptive Machine Learning for Time-Varying Systems: Low Dimensional
Latent Space Tuning [91.3755431537592]
We present a recently developed method of adaptive machine learning for time-varying systems.
Our approach is to map very high (N>100k) dimensional inputs into the low dimensional (N2) latent space at the output of the encoder section of an encoder-decoder CNN.
This method allows us to learn correlations within and to track their evolution in real time based on feedback without interrupts.
arXiv Detail & Related papers (2021-07-13T16:05:28Z) - Fast Modeling and Understanding Fluid Dynamics Systems with
Encoder-Decoder Networks [0.0]
We show that an accurate deep-learning-based proxy model can be taught efficiently by a finite-volume-based simulator.
Compared to traditional simulation, the proposed deep learning approach enables much faster forward computation.
We quantify the sensitivity of the deep learning model to key physical parameters and hence demonstrate that the inversion problems can be solved with great acceleration.
arXiv Detail & Related papers (2020-06-09T17:14:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.