TopoFormer: Integrating Transformers and ConvLSTMs for Coastal Topography Prediction

• URL: http://arxiv.org/abs/2501.06494v1
• Date: Sat, 11 Jan 2025 09:46:02 GMT
• Title: TopoFormer: Integrating Transformers and ConvLSTMs for Coastal Topography Prediction
• Authors: Santosh Munian, Oktay Karakuş, William Russell, Gwyn Nelson,
• Abstract summary: textitTopoFormer integrates transformer-based encoders with convolutional long short-term memory layers.<n>The architecture captures both long-range dependencies and localized temporal patterns inherent in beach profiles data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents \textit{TopoFormer}, a novel hybrid deep learning architecture that integrates transformer-based encoders with convolutional long short-term memory (ConvLSTM) layers for the precise prediction of topographic beach profiles referenced to elevation datums, with a particular focus on Mean Low Water Springs (MLWS) and Mean Low Water Neaps (MLWN). Accurate topographic estimation down to MLWS is critical for coastal management, navigation safety, and environmental monitoring. Leveraging a comprehensive dataset from the Wales Coastal Monitoring Centre (WCMC), consisting of over 2000 surveys across 36 coastal survey units, TopoFormer addresses key challenges in topographic prediction, including temporal variability and data gaps in survey measurements. The architecture uniquely combines multi-head attention mechanisms and ConvLSTM layers to capture both long-range dependencies and localized temporal patterns inherent in beach profiles data. TopoFormer's predictive performance was rigorously evaluated against state-of-the-art models, including DenseNet, 1D/2D CNNs, and LSTMs. While all models demonstrated strong performance, \textit{TopoFormer} achieved the lowest mean absolute error (MAE), as low as 2 cm, and provided superior accuracy in both in-distribution (ID) and out-of-distribution (OOD) evaluations.

Related papers

• Exploring the Potential of Latent Embeddings for Sea Ice Characterization using ICESat-2 Data [1.4364491422470593]
  Ice, Cloud, and Elevation Satellite-2 (ICESat-2) provides high-resolution measurements of sea ice height. Recent studies have developed machine learning methods on ICESat-2 data, primarily focusing on surface type classification. We develop autoencoder models based on Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN) to reconstruct topographic sequences from ICESat-2 and derive embeddings. Our results show that embeddings from autoencoders preserve the overall structure but generate relatively more compact clusters compared to the original ICESat-2 data.
  arXiv  Detail & Related papers  (2025-04-25T19:42:09Z)
• Temporal Graph MLP Mixer for Spatio-Temporal Forecasting [1.5696662871407674]
  Temporal Graph-Mixer (T-GMM) architecture designed to address missing data challenges. Model combines node-level processing with subgraph encoding to capture localized spatial dependencies. Experiments on AQI, ENGRAD, PV-US and METR-LA datasets demonstrate model's ability to effectively forecast even in the presence of significant missing data.
  arXiv  Detail & Related papers  (2025-01-17T14:13:48Z)
• MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling [68.69647625472464]
  Downscaling, a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions. Previous downscaling methods lacked tailored designs for meteorology and encountered structural limitations. We propose a novel model called MambaDS, which enhances the utilization of multivariable correlations and topography information.
  arXiv  Detail & Related papers  (2024-08-20T13:45:49Z)
• Deep Vision-Based Framework for Coastal Flood Prediction Under Climate Change Impacts and Shoreline Adaptations [0.3413711585591077]
  We present a systematic framework for training high-fidelity Deep Vision-based coastal flood prediction models in low-data settings. We also introduce a deep CNN architecture tailored specifically to the coastal flood prediction problem at hand. The performance of the developed DL models is validated against commonly adopted geostatistical regression methods.
  arXiv  Detail & Related papers  (2024-06-06T19:54:34Z)
• Observation-Guided Meteorological Field Downscaling at Station Scale: A Benchmark and a New Method [66.80344502790231]
  We extend meteorological downscaling to arbitrary scattered station scales and establish a new benchmark and dataset. Inspired by data assimilation techniques, we integrate observational data into the downscaling process, providing multi-scale observational priors. Our proposed method outperforms other specially designed baseline models on multiple surface variables.
  arXiv  Detail & Related papers  (2024-01-22T14:02:56Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Long-term drought prediction using deep neural networks based on geospatial weather data [75.38539438000072]
  High-quality drought forecasting up to a year in advance is critical for agriculture planning and insurance. We tackle drought data by introducing an end-to-end approach that adopts a systematic end-to-end approach. Key findings are the exceptional performance of a Transformer model, EarthFormer, in making accurate short-term (up to six months) forecasts.
  arXiv  Detail & Related papers  (2023-09-12T13:28:06Z)
• Transforming Observations of Ocean Temperature with a Deep Convolutional Residual Regressive Neural Network [0.0]
  Sea surface temperature (SST) is an essential climate variable that can be measured via ground truth, remote sensing, or hybrid model methodologies. Here, we celebrate SST surveillance progress via the application of a few relevant technological advances from the late 20th and early 21st century. We develop our existing water cycle observation framework, Flux to Flow (F2F), to fuse AMSR-E and MODIS into a higher resolution product. Our neural network architecture is constrained to a deep convolutional residual regressive neural network.
  arXiv  Detail & Related papers  (2023-06-16T17:35:11Z)
• A Generative Deep Learning Approach to Stochastic Downscaling of Precipitation Forecasts [0.5906031288935515]
  Generative adversarial networks (GANs) have been demonstrated by the computer vision community to be successful at super-resolution problems. We show that GANs and VAE-GANs can match the statistical properties of state-of-the-art pointwise post-processing methods whilst creating high-resolution, spatially coherent precipitation maps.
  arXiv  Detail & Related papers  (2022-04-05T07:19:42Z)
• DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation [50.08080424613603]
  Long-range correlation is essential for accurate monocular depth estimation. We propose to leverage the Transformer to model this global context with an effective attention mechanism. Our proposed model, termed DepthFormer, surpasses state-of-the-art monocular depth estimation methods with prominent margins.
  arXiv  Detail & Related papers  (2022-03-27T05:03:56Z)
• Uncertainty-Aware Learning for Improvements in Image Quality of the Canada-France-Hawaii Telescope [9.963669010212012]
  We leverage state-of-the-art machine learning methods to predict observatory image quality (IQ) from environmental conditions and observatory operating parameters. We develop accurate and interpretable models of the complex dependence between data features and observed IQ for CFHT's wide field camera, MegaCam.
  arXiv  Detail & Related papers  (2021-06-30T18:10:20Z)
• Unsupervised Scale-consistent Depth Learning from Video [131.3074342883371]
  We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training. Thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system. The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training.
  arXiv  Detail & Related papers  (2021-05-25T02:17:56Z)
• Numerical Weather Forecasting using Convolutional-LSTM with Attention and Context Matcher Mechanisms [10.759556555869798]
  We introduce a novel deep learning architecture for forecasting high-resolution weather data. Our Weather Model achieves significant performance improvements compared to baseline deep learning models.
  arXiv  Detail & Related papers  (2021-02-01T08:30:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.