Rethinking Inductive Bias in Geographically Neural Network Weighted Regression

• URL: http://arxiv.org/abs/2507.09958v4
• Date: Mon, 21 Jul 2025 17:15:03 GMT
• Title: Rethinking Inductive Bias in Geographically Neural Network Weighted Regression
• Authors: Zhenyuan Chen,
• Abstract summary: This work revisits the inductive biases in Geographically Neural Network Weighted Regression.<n>We introduce local receptive fields, sequential context, and self-attention into spatial regression.<n>We show that GNNWR outperforms classic methods in capturing nonlinear and complex spatial relationships.
• Score: 1.3597551064547502
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Inductive bias is a key factor in spatial regression models, determining how well a model can learn from limited data and capture spatial patterns. This work revisits the inductive biases in Geographically Neural Network Weighted Regression (GNNWR) and identifies limitations in current approaches for modeling spatial non-stationarity. While GNNWR extends traditional Geographically Weighted Regression by using neural networks to learn spatial weighting functions, existing implementations are often restricted by fixed distance-based schemes and limited inductive bias. We propose to generalize GNNWR by incorporating concepts from convolutional neural networks, recurrent neural networks, and transformers, introducing local receptive fields, sequential context, and self-attention into spatial regression. Through extensive benchmarking on synthetic spatial datasets with varying heterogeneity, noise, and sample sizes, we show that GNNWR outperforms classic methods in capturing nonlinear and complex spatial relationships. Our results also reveal that model performance depends strongly on data characteristics, with local models excelling in highly heterogeneous or small-sample scenarios, and global models performing better with larger, more homogeneous data. These findings highlight the importance of inductive bias in spatial modeling and suggest future directions, including learnable spatial weighting functions, hybrid neural architectures, and improved interpretability for models handling non-stationary spatial data.

Related papers

• Modeling Spatial Extremes using Non-Gaussian Spatial Autoregressive Models via Convolutional Neural Networks [14.37149160708975]
  We present a spatial autoregressive modeling framework, which maps observations at a location and its neighbors to independent random variables.<n>In particular, we consider the SAR model with Generalized Extreme Value distribution innovations to combine the observation at a central grid location with its neighbors.<n>We apply this model to analyze annual maximum precipitation data from ERA-Interim-driven Weather Research and Forecasting (WRF) simulations.
  arXiv  Detail & Related papers  (2025-05-05T21:26:02Z)
• Artificial Geographically Weighted Neural Network: A Novel Framework for Spatial Analysis with Geographically Weighted Layers [0.0]
  AGWNN is a novel framework that integrates geographically weighted techniques with neural networks to capture complex nonlinear spatial relationships.<n>To rigorously evaluate the performance of AGWNN, we conducted comprehensive experiments using both simulated datasets and real-world case studies.
  arXiv  Detail & Related papers  (2025-04-01T01:48:46Z)
• A Subsampling Based Neural Network for Spatial Data [0.0]
  This article proposes a consistent localized two-layer deep neural network-based regression for spatial data. We empirically observe the rate of convergence of discrepancy measures between the empirical probability distribution of observed and predicted data, which will become faster for a less smooth spatial surface. This application is an effective showcase of non-linear spatial regression.
  arXiv  Detail & Related papers  (2024-11-06T02:37:43Z)
• Neural networks for geospatial data [0.0]
  NN-GLS is a new neural network estimation algorithm for the non-linear mean in GP models. We show that NN-GLS admits a representation as a special type of graph neural network (GNN) Theoretically, we show that NN-GLS will be consistent for irregularly observed spatially correlated data processes.
  arXiv  Detail & Related papers  (2023-04-18T17:52:23Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets [57.06026574261203]
  We provide theoretical evidence for learning low-dimensional state spaces, which can also model long-term memory. Experiments corroborate our theory, demonstrating extrapolation via learning low-dimensional state spaces with both linear and non-linear RNNs.
  arXiv  Detail & Related papers  (2022-10-25T14:45:15Z)
• STONet: A Neural-Operator-Driven Spatio-temporal Network [38.5696882090282]
  Graph-based graph-temporal neural networks are effective to model spatial dependency among discrete points sampled irregularly. We propose atemporal framework based on neural operators for PDEs, which learn the mechanisms governing the dynamics of spatially-continuous physical quantities. Experiments show our model's performance on forecasting spatially-continuous physic quantities, and its superior to unseen spatial points and ability to handle temporally-irregular data.
  arXiv  Detail & Related papers  (2022-04-18T17:20:12Z)
• Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs [77.33781731432163]
  We learn dynamic graph representation in hyperbolic space, for the first time, which aims to infer node representations. We present a novel Hyperbolic Variational Graph Network, referred to as HVGNN. In particular, to model the dynamics, we introduce a Temporal GNN (TGNN) based on a theoretically grounded time encoding approach.
  arXiv  Detail & Related papers  (2021-04-06T01:44:15Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Bayesian Graph Neural Networks with Adaptive Connection Sampling [62.51689735630133]
  We propose a unified framework for adaptive connection sampling in graph neural networks (GNNs) The proposed framework not only alleviates over-smoothing and over-fitting tendencies of deep GNNs, but also enables learning with uncertainty in graph analytic tasks with GNNs.
  arXiv  Detail & Related papers  (2020-06-07T07:06:35Z)
• Stochastic Graph Neural Networks [123.39024384275054]
  Graph neural networks (GNNs) model nonlinear representations in graph data with applications in distributed agent coordination, control, and planning. Current GNN architectures assume ideal scenarios and ignore link fluctuations that occur due to environment, human factors, or external attacks. In these situations, the GNN fails to address its distributed task if the topological randomness is not considered accordingly.
  arXiv  Detail & Related papers  (2020-06-04T08:00:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.