Efficient Temporal Modeling for Mobile Sleep Staging via Lightweight Random Attention

• URL: http://arxiv.org/abs/2606.13694v1
• Date: Sun, 31 May 2026 05:12:26 GMT
• Title: Efficient Temporal Modeling for Mobile Sleep Staging via Lightweight Random Attention
• Abstract summary: We propose Random Attention (RA), a lightweight temporal modeling module based on fixed random projections.<n> RA introduces little additional parameters beyond the epoch encoder while enabling effective temporal smoothing.<n>Experiments on Sleep-EDF-20 and Sleep-EDF-78 show that RA consistently improves epoch-wise baselines by 1-3% in accuracy and F1 score.
• Score: 11.02990137178079
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Mobile sleep staging serves as a foundational infrastructure for in-home sleep monitoring and closed-loop modulation. But existing sequential models such as RNNs and Transformers are computationally expensive for mobile deployment. In this paper, we propose Random Attention (RA), a lightweight temporal modeling module based on fixed random projections, which replaces learnable sequence modeling with similarity-based aggregation. RA introduces little additional parameters beyond the epoch encoder while enabling effective temporal smoothing. We further provide a theoretical interpretation via the Random Attention Prior Kernel (RAPK), which decomposes RA into a global smoothing term and a feature similarity term, offering an interpretable view of temporal sleep structure. Experiments on Sleep-EDF-20 and Sleep-EDF-78 show that RA consistently improves epoch-wise baselines by 1-3\% in accuracy and F1 score, while achieving competitive performance compared with LSTM, GRU, and Transformer models. RA also demonstrates strong generalization across different backbone encoders and improved robustness over conventional temporal smoothing methods. These results indicate that efficient sleep staging can be achieved through lightweight similarity-based temporal aggregation, making RA suitable for real-time wearable applications.

Related papers

• NanoSleep: A Parameter-Efficient Hybrid Temporal Convolutional Network for Single-Channel Sleep Stage Classification [0.0]
  We present NanoSleep, a compact hybrid temporal convolutional network for automatic sleep stage classification.<n>NanoSleep provides an effective balance between accuracy and efficiency, making it well suited for wearable devices, home-based sleep monitoring, and resource-constrained clinical applications.
  arXiv  Detail & Related papers  (2026-08-19T06:08:24Z)
• SWINSleepNet: A Hierarchical Context-Aware Framework for Sleep Staging (v2) [3.911341147858216]
  Traditional epoch-level encoders fail to extract subtle temporal microstructures and intra-epoch cross-region interactions.<n>We propose SwinSleepNet, a hierarchical context-aware dual-stream framework that separately optimize intra-epoch representation learning and inter-epoch contextual modeling.<n>Our method achieves competitive overall performance, and exhibits stronger robustness and stability on difficult N1 stages and transitional epochs.
  arXiv  Detail & Related papers  (2026-08-03T13:07:49Z)
• Rethinking Random Transformers as Adaptive Sequence Smoothers for Sleep Staging [17.920922817772624]
  We show that a randomly Kernel Transformer, without any training, substantially improves sleep staging performance.<n>We formalize this effect via a Random Attention Prior (RAPK), showing that random self-attention acts as an adaptive smoother.
  arXiv  Detail & Related papers  (2026-05-11T02:48:06Z)
• Perceive, Route and Modulate: Dynamic Pattern Recalibration for Time Series Forecasting [68.53528085362642]
  Current deep forecasting models rely on fixed weight matrices applied uniformly to all temporal tokens.<n>We introduce Dynamic Pattern Recalibration, a backbone-agnostic mechanism that resolves this via token-level recalibration.<n>DPRNet achieves competitive performance across 12 benchmarks, validating dynamic recalibration against macroscopic parameter scaling.
  arXiv  Detail & Related papers  (2026-05-07T14:12:47Z)
• Preserving Temporal Dynamics in Time Series Generation [0.2099922236065961]
  Time-series data augmentation plays a crucial role in regression-oriented forecasting tasks.<n>Existing approaches primarily focus on matching marginal data distributions.<n>We propose a model-agnostic Markov Chain Monte Carlo framework to mitigate distribution shift and preserve temporal dynamics.
  arXiv  Detail & Related papers  (2026-04-29T20:32:45Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• Low-Dimensional Adaptation of Rectified Flow: A New Perspective through the Lens of Diffusion and Stochastic Localization [59.04314685837778]
  Rectified flow (RF) has gained considerable popularity due to its generation efficiency and state-of-the-art performance.<n>In this paper, we investigate the degree to which RF automatically adapts to the intrinsic low dimensionality of the support of the target distribution to accelerate sampling.<n>We show that, using a carefully designed choice of the time-discretization scheme and with sufficiently accurate drift estimates, the RF sampler enjoys an complexity of order $O(k/varepsilon)$.
  arXiv  Detail & Related papers  (2026-01-21T22:09:27Z)
• A Context-Aware Temporal Modeling through Unified Multi-Scale Temporal Encoding and Hierarchical Sequence Learning for Single-Channel EEG Sleep Staging [0.0]
  This study focuses on single-channel electroencephalography (EEG), a practical and widely available signal for automatic sleep staging.<n>Existing approaches face challenges such as class imbalance, limited receptive-field modeling, and insufficient interpretability.<n>This work proposes a context-aware and interpretable framework for single-channel EEG sleep staging.
  arXiv  Detail & Related papers  (2025-12-28T15:42:33Z)
• Leveraging Duration Pseudo-Embeddings in Multilevel LSTM and GCN Hypermodels for Outcome-Oriented PPM [4.120576565537633]
  Existing deep learning models for Predictive Process Monitoring (PPM) struggle with temporal irregularities.<n>We propose a dual input neural network strategy that separates event and sequence attributes, using a duration-aware pseudo-embedding matrix.<n>Our results demonstrate the benefits of explicit temporal encoding and provide a flexible design for robust, real-world PPM applications.
  arXiv  Detail & Related papers  (2025-11-24T07:06:08Z)
• TIMED: Adversarial and Autoregressive Refinement of Diffusion-Based Time Series Generation [0.31498833540989407]
  TIMED is a unified generative framework that captures global structure via a forward-reverse diffusion process.<n>To further align the real and synthetic distributions in feature space, TIMED incorporates a Maximum Mean Discrepancy (MMD) loss.<n>We show that TIMED generates more realistic and temporally coherent sequences than state-of-the-art generative models.
  arXiv  Detail & Related papers  (2025-09-23T23:05:40Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.