Evaluating Transformer and LSTM Frameworks for Prediction in Ungauged Basins

• URL: http://arxiv.org/abs/2606.02791v1
• Date: Mon, 01 Jun 2026 18:57:20 GMT
• Title: Evaluating Transformer and LSTM Frameworks for Prediction in Ungauged Basins
• Abstract summary: In ungauged basins, the absence of direct observations increases uncertainty and limits the ability to anticipate extreme events.<n>This study evaluates whether an encoder-only Transformer provides an advantage over an LSTM for upstream streamflow inference under limited hydrologic information.
• Score: 4.077787659104315
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Watershed networks exhibit convergent topologies in which multiple tributaries merge into downstream channels,integrating diverse upstream hydrological processes. In ungauged basins, the absence of direct observations increases uncertainty and limits the ability to anticipate extreme events. This study evaluates whether an encoder-only Transformer provides an advantage over an LSTM for upstream streamflow inference under limited hydrologic information, using retrospective simulations from the NOAA National Water Model (NWM). Across both upstream-only and combined configurations, the LSTM showed stronger overall performance than the Transformer model across the two configurations. Incorporating downstream information further boosted performance for all models, increasing median NNSE by more than 60%. Rather than treating this as a leaderboard-style comparison, we interpret the experiments as a test of architectural inductive bias for hydrologic sequence inference. The results indicate that recurrent memory remains better aligned with this upstream reconstruction task than an encoder-only Transformer, while downstream hydrologic context provides a strong auxiliary constraint that substantially improves prediction skill across architectures

Related papers

• DUET -- Dual User Embedding Transformers for Offsite Conversion Prediction [8.972672149050627]
  Offsite conversion rate (OCVR) prediction is an important ranking problem in computational recommendation systems.<n>This task presents a modeling challenge: click signals are abundant and exhibit short temporal horizons, whereas conversion signals are inherently sparse, long-delayed, and frequently unattributed.<n>We propose DUET, a framework that explicitly partitions user behavioral data into two domain-coherent streams -- clicks and conversions.
  arXiv  Detail & Related papers  (2026-06-08T23:13:58Z)
• High-fidelity Modeling of Full-scale Pressurized Water Reactor Flow Fields for Machine Learning Applications [3.8347851078346427]
  This work presents a high-fidelity computational fluid dynamics (CFD) and data-driven modeling framework for assembly-level flow characterization in a pressurized water reactor.<n>A full lower-plenum and core-inlet domain was constructed using publicly available geometry and operating conditions, enabling transient simulations with pump-induced swirl boundary conditions.<n>Results show that cold-leg swirl and lower-plenum transport generate strongly heterogeneous assembly-wise inlet flow distributions, particularly near the lower core region.<n>A 3D convolutional-based inpainting model successfully recon-structed missing assembly-level mass flow rates from partial observations
  arXiv  Detail & Related papers  (2026-05-23T22:50:23Z)
• Learning on the Manifold: Unlocking Standard Diffusion Transformers with Representation Encoders [48.68968421120471]
  We show that standard diffusion transformers fail to converge on representations directly.<n>We identify Geometric Interference as the root cause.<n>Our method RJF enables the standard DiT-B architecture to converge effectively, achieving an FID of 3.37.
  arXiv  Detail & Related papers  (2026-02-10T18:58:04Z)
• Diffusion Bridge or Flow Matching? A Unifying Framework and Comparative Analysis [57.614436689939986]
  Diffusion Bridge and Flow Matching have both demonstrated compelling empirical performance in transformation between arbitrary distributions.<n>We recast their frameworks through the lens of Optimal Control and prove that the cost function of the Diffusion Bridge is lower.<n>To corroborate these theoretical claims, we propose a novel, powerful architecture for Diffusion Bridge built on a latent Transformer.
  arXiv  Detail & Related papers  (2025-09-29T09:45:22Z)
• Wavelet-SARIMA-Transformer: A Hybrid Model for Rainfall Forecasting [0.0]
  This study develops and evaluates a novel hybridWavelet SARIMA Transformer, WST framework to forecast using monthly rainfall across five meteorological subdivisions of Northeast India over the 1971 to 2023 period.
  arXiv  Detail & Related papers  (2025-09-15T13:27:19Z)
• FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems [51.15230303652732]
  FLEX (F Low EXpert) is a backbone architecture for generative modeling of-temporal physical systems.<n>It reduces the variance of the velocity field in the diffusion model, which helps stabilize training.<n>It achieves accurate predictions for super-resolution and forecasting tasks using as few features as two reverse diffusion steps.
  arXiv  Detail & Related papers  (2025-05-23T00:07:59Z)
• Non-asymptotic Convergence of Training Transformers for Next-token Prediction [48.9399496805422]
  Transformers have achieved extraordinary success in modern machine learning due to their excellent ability to handle sequential data. This paper provides a fine-grained non-asymptotic analysis of the training dynamics of a one-layer transformer. We show that the trained transformer presents non-token prediction ability with dataset shift.
  arXiv  Detail & Related papers  (2024-09-25T20:22:06Z)
• Towards Generalized Hydrological Forecasting using Transformer Models for 120-Hour Streamflow Prediction [0.34530027457862006]
  This study explores the efficacy of a Transformer model for 120-hour streamflow prediction across 125 diverse locations in Iowa, US. We benchmarked the Transformer model's performance against three deep learning models (LSTM, GRU, and Seq2Seq) and the Persistence approach. The study reveals the Transformer model's superior performance, maintaining higher median NSE and KGE scores and exhibiting the lowest NRMSE values.
  arXiv  Detail & Related papers  (2024-06-11T17:26:14Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• Probing the limit of hydrologic predictability with the Transformer network [7.326504492614808]
  We show that a vanilla Transformer architecture is not competitive against LSTM on the widely benchmarked CAMELS dataset. A recurrence-free variant of Transformer can obtain mixed comparisons with LSTM, producing the same Kling-Gupta efficiency coefficient (KGE) along with other metrics. While the Transformer results are not higher than current state-of-the-art, we still learned some valuable lessons.
  arXiv  Detail & Related papers  (2023-06-21T17:06:54Z)
• Temporal Fusion Transformers for Streamflow Prediction: Value of Combining Attention with Recurrence [0.0]
  This work tests the hypothesis that combining recurrence with attention can improve streamflow prediction. We set up the Temporal Fusion Transformer (TFT) architecture, a model that combines both of these aspects and has never been applied in hydrology before. Our results demonstrate that TFT indeed exceeds the performance benchmark set by the LSTM and Transformers for streamflow prediction.
  arXiv  Detail & Related papers  (2023-05-21T03:58:16Z)
• GMFlow: Learning Optical Flow via Global Matching [124.57850500778277]
  We propose a GMFlow framework for learning optical flow estimation. It consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. Our new framework outperforms 32-iteration RAFT's performance on the challenging Sintel benchmark.
  arXiv  Detail & Related papers  (2021-11-26T18:59:56Z)
• What Matters in Unsupervised Optical Flow [51.45112526506455]
  We compare and analyze a set of key components in unsupervised optical flow. We construct a number of novel improvements to unsupervised flow models. We present a new unsupervised flow technique that significantly outperforms the previous state-of-the-art.
  arXiv  Detail & Related papers  (2020-06-08T19:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.