Sleep Deprivation in the Forward-Forward Algorithm

• URL: http://arxiv.org/abs/2310.18647v1
• Date: Sat, 28 Oct 2023 09:09:44 GMT
• Title: Sleep Deprivation in the Forward-Forward Algorithm
• Abstract summary: We show the size of the gap between the sleep and awake phase influences the learning capabilities of the algorithm. We highlight the importance of negative data in diminishing the devastating effects of sleep deprivation.
• Score: 2.7195102129095003
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper aims to explore the separation of the two forward passes in the Forward-Forward algorithm from a biological perspective in the context of sleep. We show the size of the gap between the sleep and awake phase influences the learning capabilities of the algorithm and highlight the importance of negative data in diminishing the devastating effects of sleep deprivation.

Related papers

• Minimizing the Effect of Sleep Deprivation in the Forward-Forward Algorithm [1.2752808844888015]
  Previous research has demonstrated that sleep deprivation in the Forward-Forward algorithm has a catastrophic effect on learning efficacy.<n>To mitigate this issue, we explore several approaches; these include alternative activation, optimized loss function, and threshold tuning.<n>We additionally investigate the potential of caffeine-induced stimulation to enhance performance during sleep-deprived conditions.
  arXiv  Detail & Related papers  (2026-09-05T11:54:27Z)
• Annotation of Sleep Depth Index with Scalable Deep Learning Yields Novel Digital Biomarkers for Sleep Health [15.197165195697666]
  Traditional sleep staging categorizes sleep and wakefulness into five coarse-grained classes, overlooking subtle variations within each stage. We propose a deep-learning method for automatic and scalable annotation of sleep depth index using existing sleep staging labels. Our approach is validated using polysomnography from over ten thousand recordings across four large-scale cohorts.
  arXiv  Detail & Related papers  (2024-07-05T08:42:49Z)
• ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularization [52.5587113539404]
  We introduce a causality-aware entropy term that effectively identifies and prioritizes actions with high potential impacts for efficient exploration. Our proposed algorithm, ACE: Off-policy Actor-critic with Causality-aware Entropy regularization, demonstrates a substantial performance advantage across 29 diverse continuous control tasks.
  arXiv  Detail & Related papers  (2024-02-22T13:22:06Z)
• SI-SD: Sleep Interpreter through awake-guided cross-subject Semantic Decoding [5.283755248013948]
  We design a novel cognitive neuroscience experiment and collect a comprehensive, well-annotated electroencephalography (EEG) dataset from 134 subjects during both wakefulness and sleep. We develop SI-SD that enhances sleep semantic decoding through the position-wise alignment of neural latent sequence between wakefulness and sleep.
  arXiv  Detail & Related papers  (2023-09-28T14:06:34Z)
• Contrastive Learning for Sleep Staging based on Inter Subject Correlation [2.092935059855151]
  This paper presents the MViTime model that have been used in sleep staging study. We implement the inter-subject correlation theory through contrastive learning to address the cross-subject problem in sleep stage classification.
  arXiv  Detail & Related papers  (2023-05-05T12:16:01Z)
• Sleep Activity Recognition and Characterization from Multi-Source Passively Sensed Data [67.60224656603823]
  Sleep Activity Recognition methods can provide indicators to assess, monitor, and characterize subjects' sleep-wake cycles and detect behavioral changes. We propose a general method that continuously operates on passively sensed data from smartphones to characterize sleep and identify significant sleep episodes. Thanks to their ubiquity, these devices constitute an excellent alternative data source to profile subjects' biorhythms in a continuous, objective, and non-invasive manner.
  arXiv  Detail & Related papers  (2023-01-17T15:18:45Z)
• ProductGraphSleepNet: Sleep Staging using Product Spatio-Temporal Graph Learning with Attentive Temporal Aggregation [4.014524824655106]
  This work proposes an adaptive product graph learning-based graph convolutional network, named ProductGraphSleepNet, for learning joint-temporal graphs. The proposed network makes it possible for clinicians to comprehend and interpret the learned connectivity graphs for sleep stages.
  arXiv  Detail & Related papers  (2022-12-09T14:34:58Z)
• Heterogeneous Hidden Markov Models for Sleep Activity Recognition from Multi-Source Passively Sensed Data [67.60224656603823]
  Psychiatric patients' passive activity monitoring is crucial to detect behavioural shifts in real-time. Sleep Activity Recognition constitutes a behavioural marker to portray patients' activity cycles. Mobile passively sensed data captured from smartphones constitute an excellent alternative to profile patients' biorhythm.
  arXiv  Detail & Related papers  (2022-11-08T17:29:40Z)
• Continual learning benefits from multiple sleep mechanisms: NREM, REM, and Synaptic Downscaling [51.316408685035526]
  Learning new tasks and skills in succession without losing prior learning is a computational challenge for both artificial and biological neural networks. Here, we investigate how modeling three distinct components of mammalian sleep together affects continual learning in artificial neural networks.
  arXiv  Detail & Related papers  (2022-09-09T13:45:27Z)
• TransSleep: Transitioning-aware Attention-based Deep Neural Network for Sleep Staging [2.105172041656126]
  We propose a novel deep neural network structure, TransSleep, that captures distinctive local temporal patterns. Results show that TransSleep achieves promising performance in automatic sleep staging.
  arXiv  Detail & Related papers  (2022-03-22T08:55:32Z)
• Disentangling Adaptive Gradient Methods from Learning Rates [65.0397050979662]
  We take a deeper look at how adaptive gradient methods interact with the learning rate schedule. We introduce a "grafting" experiment which decouples an update's magnitude from its direction. We present some empirical and theoretical retrospectives on the generalization of adaptive gradient methods.
  arXiv  Detail & Related papers  (2020-02-26T21:42:49Z)
• The Break-Even Point on Optimization Trajectories of Deep Neural Networks [64.7563588124004]
  We argue for the existence of the "break-even" point on this trajectory. We show that using a large learning rate in the initial phase of training reduces the variance of the gradient. We also show that using a low learning rate results in bad conditioning of the loss surface even for a neural network with batch normalization layers.
  arXiv  Detail & Related papers  (2020-02-21T22:55:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.