Space-Filling Regularization for Robust and Interpretable Nonlinear State Space Models

• URL: http://arxiv.org/abs/2507.07792v1
• Date: Thu, 10 Jul 2025 14:19:29 GMT
• Title: Space-Filling Regularization for Robust and Interpretable Nonlinear State Space Models
• Authors: Hermann Klein, Max Heinz Herkersdorf, Oliver Nelles,
• Abstract summary: During training, the state trajectory can deform significantly leading to poor data coverage of the state space.<n>This can cause significant issues for space-oriented training algorithms.<n>This paper proposes a new type of space-filling regularization that ensures a favorable data distribution in state space.
• Score: 0.16385815610837165
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The state space dynamics representation is the most general approach for nonlinear systems and often chosen for system identification. During training, the state trajectory can deform significantly leading to poor data coverage of the state space. This can cause significant issues for space-oriented training algorithms which e.g. rely on grid structures, tree partitioning, or similar. Besides hindering training, significant state trajectory deformations also deteriorate interpretability and robustness properties. This paper proposes a new type of space-filling regularization that ensures a favorable data distribution in state space via introducing a data-distribution-based penalty. This method is demonstrated in local model network architectures where good interpretability is a major concern. The proposed approach integrates ideas from modeling and design of experiments for state space structures. This is why we present two regularization techniques for the data point distributions of the state trajectories for local affine state space models. Beyond that, we demonstrate the results on a widely known system identification benchmark.

Related papers

• Fine-grained Spatio-temporal Event Prediction with Self-adaptive Anchor Graph [8.435985634889285]
  We propose a novel Graph Spatio-Temporal Point ( GSTPP) model for fine-grained event prediction.<n>It adopts an encoder-coder architecture that jointly models the state dynamics of spatially localized regions.<n>The proposed GSTPP model greatly improves the accuracy of fine-grained event prediction.
  arXiv  Detail & Related papers  (2025-01-15T08:38:07Z)
• LISA: Learning-Integrated Space Partitioning Framework for Traffic Accident Forecasting on Heterogeneous Spatiotemporal Data [14.726248469735971]
  Traffic accident forecasting is an important task for intelligent transportation management and emergency response systems.<n>Existing data-driven methods fail to handle the heterogeneous accident patterns over space at different scales.<n>We propose a novel Learning-Integrated Space Partition Framework (LISA) to simultaneously learn partitions while training models.
  arXiv  Detail & Related papers  (2024-12-19T19:52:19Z)
• Reconstructing Spatiotemporal Data with C-VAEs [49.1574468325115]
  Conditional continuous representation of moving regions is commonly used. In this work, we explore the capabilities of Conditional Varitemporal Autoencoder (C-VAE) models to generate realistic representations of regions' evolution.
  arXiv  Detail & Related papers  (2023-07-12T15:34:10Z)
• Deep Spatial Domain Generalization [8.102110157532556]
  We develop the spatial graph neural network that handles spatial data as a graph and learns the spatial embedding on each node. The proposed method infers the spatial embedding of an unseen location during the test phase and decodes the parameters of the downstream-task model directly on the target location.
  arXiv  Detail & Related papers  (2022-10-03T06:16:20Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• Decentralized Vehicle Coordination: The Berkeley DeepDrive Drone Dataset and Consensus-Based Models [76.32775745488073]
  We present a novel dataset and modeling framework designed to study motion planning in understructured environments.<n>We demonstrate that a consensus-based modeling approach can effectively explain the emergence of priority orders observed in our dataset.
  arXiv  Detail & Related papers  (2022-09-19T05:06:57Z)
• Spatial Entropy Regularization for Vision Transformers [71.44392961125807]
  Vision Transformers (VTs) can contain a semantic segmentation structure which does not spontaneously emerge when training is supervised. We propose a VT regularization method based on a spatial formulation of the information entropy. We show that the proposed regularization approach is beneficial with different training scenarios, datasets, downstream tasks and VT architectures.
  arXiv  Detail & Related papers  (2022-06-09T17:34:39Z)
• Cyclic Graph Attentive Match Encoder (CGAME): A Novel Neural Network For OD Estimation [8.398623478484248]
  Origin-Destination Estimation plays an important role in traffic management and traffic simulation in the era of Intelligent Transportation System (ITS) Previous model-based models face the under-determined challenge, thus desperate demand for additional assumptions and extra data exists. We propose Cyclic Graph Attentive Matching (C-GAME) based on a novel Graph Matcher with double-layer attention mechanism.
  arXiv  Detail & Related papers  (2021-11-26T08:57:21Z)
• Sign-Agnostic CONet: Learning Implicit Surface Reconstructions by Sign-Agnostic Optimization of Convolutional Occupancy Networks [39.65056638604885]
  We learn implicit surface reconstruction by sign-agnostic optimization of convolutional occupancy networks. We show this goal can be effectively achieved by a simple yet effective design.
  arXiv  Detail & Related papers  (2021-05-08T03:35:32Z)
• Localized active learning of Gaussian process state space models [63.97366815968177]
  A globally accurate model is not required to achieve good performance in many common control applications. We propose an active learning strategy for Gaussian process state space models that aims to obtain an accurate model on a bounded subset of the state-action space. By employing model predictive control, the proposed technique integrates information collected during exploration and adaptively improves its exploration strategy.
  arXiv  Detail & Related papers  (2020-05-04T05:35:02Z)
• Discrete Variational Attention Models for Language Generation [51.88612022940496]
  We propose a discrete variational attention model with categorical distribution over the attention mechanism owing to the discrete nature in languages. Thanks to the property of discreteness, the training of our proposed approach does not suffer from posterior collapse.
  arXiv  Detail & Related papers  (2020-04-21T05:49:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.