Deep Random Features for Scalable Interpolation of Spatiotemporal Data

• URL: http://arxiv.org/abs/2412.11350v1
• Date: Mon, 16 Dec 2024 00:32:04 GMT
• Title: Deep Random Features for Scalable Interpolation of Spatiotemporal Data
• Authors: Weibin Chen, Azhir Mahmood, Michel Tsamados, So Takao,
• Abstract summary: Rapid growth of earth observation systems calls for scalable approach to interpolate remote-sensing observations. Gaussian processes (GPs) are candidate model choices for scalable interpolate. GPs usually rely on inducing points for inference, which restricts their expressivity. Deep GPs can overcome this issue, training and making inference with them are difficult.
• Score: 1.7785055672803547
• License:
• Abstract: The rapid growth of earth observation systems calls for a scalable approach to interpolate remote-sensing observations. These methods in principle, should acquire more information about the observed field as data grows. Gaussian processes (GPs) are candidate model choices for interpolation. However, due to their poor scalability, they usually rely on inducing points for inference, which restricts their expressivity. Moreover, commonly imposed assumptions such as stationarity prevents them from capturing complex patterns in the data. While deep GPs can overcome this issue, training and making inference with them are difficult, again requiring crude approximations via inducing points. In this work, we instead approach the problem through Bayesian deep learning, where spatiotemporal fields are represented by deep neural networks, whose layers share the inductive bias of stationary GPs on the plane/sphere via random feature expansions. This allows one to (1) capture high frequency patterns in the data, and (2) use mini-batched gradient descent for large scale training. We experiment on various remote sensing data at local/global scales, showing that our approach produce competitive or superior results to existing methods, with well-calibrated uncertainties.

Related papers

• Streaming Gaussian Dirichlet Random Fields for Spatial Predictions of High Dimensional Categorical Observations [0.0]
  We present a novel approach to a stream oftemporally distributed, sparse, high-dimensional categorical observations. The proposed approach efficiently learns global local patterns in Streaming data. We demonstrate the ability of a network approach to make more accurate predictions.
  arXiv  Detail & Related papers  (2024-02-23T14:52:05Z)
• NeuralGF: Unsupervised Point Normal Estimation by Learning Neural Gradient Function [55.86697795177619]
  Normal estimation for 3D point clouds is a fundamental task in 3D geometry processing. We introduce a new paradigm for learning neural gradient functions, which encourages the neural network to fit the input point clouds. Our excellent results on widely used benchmarks demonstrate that our method can learn more accurate normals for both unoriented and oriented normal estimation tasks.
  arXiv  Detail & Related papers  (2023-11-01T09:25:29Z)
• Gradient-Based Feature Learning under Structured Data [57.76552698981579]
  In the anisotropic setting, the commonly used spherical gradient dynamics may fail to recover the true direction. We show that appropriate weight normalization that is reminiscent of batch normalization can alleviate this issue. In particular, under the spiked model with a suitably large spike, the sample complexity of gradient-based training can be made independent of the information exponent.
  arXiv  Detail & Related papers  (2023-09-07T16:55:50Z)
• Deep Double Descent via Smooth Interpolation [2.141079906482723]
  We quantify sharpness of fit of training data by studying the loss landscape w.r.t. to the input variable locally to each training point. Our findings show that loss sharpness in the input space follows both model- and epoch-wise double descent, with worse peaks observed around noisy targets. While small interpolating models sharply fit both clean and noisy data, large interpolating models express a smooth loss landscape, in contrast to existing intuition.
  arXiv  Detail & Related papers  (2022-09-21T02:46:13Z)
• Inducing Gaussian Process Networks [80.40892394020797]
  We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points. The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains. We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-04-21T05:27:09Z)
• Super-resolution GANs of randomly-seeded fields [68.8204255655161]
  We propose a novel super-resolution generative adversarial network (GAN) framework to estimate field quantities from random sparse sensors. The algorithm exploits random sampling to provide incomplete views of the high-resolution underlying distributions. The proposed technique is tested on synthetic databases of fluid flow simulations, ocean surface temperature distributions measurements, and particle image velocimetry data.
  arXiv  Detail & Related papers  (2022-02-23T18:57:53Z)
• Residual Overfit Method of Exploration [78.07532520582313]
  We propose an approximate exploration methodology based on fitting only two point estimates, one tuned and one overfit. The approach drives exploration towards actions where the overfit model exhibits the most overfitting compared to the tuned model. We compare ROME against a set of established contextual bandit methods on three datasets and find it to be one of the best performing.
  arXiv  Detail & Related papers  (2021-10-06T17:05:33Z)
• Combining Pseudo-Point and State Space Approximations for Sum-Separable Gaussian Processes [48.64129867897491]
  We show that there is a simple and elegant way to combine pseudo-point methods with the state space GP approximation framework to get the best of both worlds. We demonstrate that the combined approach is more scalable and applicable to a greater range of epidemiology--temporal problems than either method on its own.
  arXiv  Detail & Related papers  (2021-06-18T16:30:09Z)
• Inter-domain Deep Gaussian Processes [45.28237107466283]
  We propose an extension of inter-domain shallow GPs that combines the advantages of inter-domain and deep Gaussian processes (DGPs) We demonstrate how to leverage existing approximate inference methods to perform simple and scalable approximate inference using inter-domain features in DGPs.
  arXiv  Detail & Related papers  (2020-11-01T04:03:35Z)
• Locally induced Gaussian processes for large-scale simulation experiments [0.0]
  We show how placement of inducing points and their multitude can be thwarted by pathologies. Our proposed methodology hybridizes global inducing point and data subset-based local GP approximation. We show that local inducing points extend their global and data-subset component parts on the accuracy--computational efficiency frontier.
  arXiv  Detail & Related papers  (2020-08-28T21:37:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.