LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning

• URL: http://arxiv.org/abs/2607.00958v1
• Date: Wed, 01 Jul 2026 13:56:21 GMT
• Title: LeNEPA: No-Augmentation Next-Latent Prediction for Time-Series Representation Learning
• Abstract summary: We study how an SSL recipe behaves when its method-specific configuration is reused unchanged after the pretraining signal family changes.<n>We introduce Latent Euclidean Next-Embedding Prediction Architecture (LeNEPA)<n>LeNEPA replaces the stop-gradient/EMA stabilization used by vanilla NEPA with SIGReg-based isotropy regularization.
• Score: 59.34593956287438
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time series are central to modern data mining applications, from industrial telemetry and server metrics to finance and physiology, yet time-series self-supervised learning often depends on view and augmentation choices that encode domain-specific invariances. We study how an SSL recipe behaves when its method-specific configuration is reused unchanged after the pretraining signal family changes, framing this as a fixed-recipe stress test rather than a comparison against optimally tuned methods. We introduce Latent Euclidean Next-Embedding Prediction Architecture (LeNEPA), a no-augmentation next-latent-token objective with a causal backbone. LeNEPA replaces the stop-gradient/EMA stabilization used by vanilla NEPA with SIGReg-based isotropy regularization and computes the predictive loss in a lightweight projected space that is discarded for evaluation. We compare LeNEPA with an ECG-tuned JEPA recipe under a fixed-horizon frozen-probe protocol on PTB-XL and Diag, a synthetic diagnostic corpus generated with Aionoscope. Both methods are retrained independently on each dataset while keeping their method-specific recipes unchanged. In this protocol, the ECG-tuned JEPA recipe is strong in-domain on PTB-XL but weaker when reused unchanged on Diag, whereas LeNEPA preserves useful frozen-probe gains on both datasets. Learning curves suggest faster early representation acquisition: LeNEPA reaches 80% of its final AUROC/AUPRC gain after 2--5k updates, compared with 5--10k updates for the faster JEPA readout. As a separate external frozen-encoder check, a CauKer-pretrained LeNEPA variant reaches 77.65% mean UCR-128 Random-Forest accuracy in a single-seed, best-checkpoint run, within 1.16 points of Mantis and within 0.24 points of MOMENT (77.89%). Overall, the results support no-augmentation latent prediction as a useful candidate recipe for low-retuning time-series SSL.

Related papers

• Self-Supervised Pretraining of Molecular Graph Encoders with LeJEPA [0.0]
  Self-supervised pretraining has transformed language and vision, but its value for molecular graph neural networks remains contested.<n>We ask whether pretraining on a large unsupervised corpus improves molecular property prediction.<n>Pretraining improves learned representations but does not robustly improve finetuning.
  arXiv  Detail & Related papers  (2026-09-02T12:08:23Z)
• SS-ESOAP: Self-Scaled Adaptive Preconditioning for Physics-Informed Learning [70.71795707610961]
  method augments SOAP-style preconditioning with a scalar secant-energy correction adapted to Kronecker geometry.<n>Method attains the lowest final residual on six of eight PDE benchmarks.<n>Three-seed $L2$ and $H1$ errors on four representative PDEs support the link between lower residuals and improved solution accuracy.
  arXiv  Detail & Related papers  (2026-08-29T21:34:47Z)
• SagnacAssisted Enhanced OTDR for Distributed Acoustic Sensing: A Standardized Benchmark and Engineering Evaluation Framework [3.5435442578622127]
  This study develops a Sagnac-assisted enhanced $$-OTDR sensing architecture and a standardized benchmark framework for engineering-oriented DAS event recognition.<n>The Sagnac interferometer provides a continuous phase response that supplements fading-prone observations in the $$-OTDR channel.<n>Experiments on a 10-km sensing fiber with six representative acoustic classes show that the dual-branch fusion model provides the most favorable trade-off among the evaluated methods.
  arXiv  Detail & Related papers  (2026-06-04T06:29:25Z)
• Uncovering Trajectory and Topological Signatures in Multimodal Pediatric Sleep Embeddings [2.2917707112773593]
  This work investigates session-wide diagnostic information contained in 30-second pediatric PSG epochs embedded by an autoencoder.<n>We test whether augmenting embeddings with PHATE-derived per-epoch coordinates and whole-night movement descriptors, persistent homology summaries of the embedding cloud, and EHR yields task-relevant signals.
  arXiv  Detail & Related papers  (2026-05-13T22:11:28Z)
• Drug Release Modeling using Physics-Informed Neural Networks [0.8390144474808291]
  We propose a novel approach using Physics-Informed Neural Networks (PINNs) and Bayesian PINNs (BPINNs)<n>By combining physical laws with experimental data, the proposed framework yields highly accurate long-term release predictions from short-term measurements.
  arXiv  Detail & Related papers  (2026-02-10T16:51:50Z)
• Ultra-Light Test-Time Adaptation for Vision--Language Models [0.6816905600359814]
  Vision-Language Models (VLMs) such as CLIP achieve strong zero-shot recognition by comparing image embeddings to text-derived class prototypes.<n>Under domain shift, they suffer from feature drift, class-prior mismatch, and severe miscalibration.<n>We propose Ultra-Light Test-Time Adaptation (UL-TTA), a fully training-free and backprop-free framework that freezes the backbone and adapts only logit-level parameters.
  arXiv  Detail & Related papers  (2025-11-12T08:19:57Z)
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics [53.247652209132376]
  Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D.<n>We present a comprehensive theory of JEPAs and instantiate it in bf LeJEPA, a lean, scalable, and theoretically grounded training objective.
  arXiv  Detail & Related papers  (2025-11-11T18:21:55Z)
• Gaussian Embeddings: How JEPAs Secretly Learn Your Data Density [51.15085346971361]
  Joint Embedding Predictive Architectures (JEPAs) learn representations able to solve numerous downstream tasks out-of-the-box.<n>JEPAs combine two objectives: (i) a latent-space prediction term, i.e., the representation of a slightly perturbed sample must be predictable from the original sample's representation, and (ii) an anti-collapse term, i.e., not all samples should have the same representation.
  arXiv  Detail & Related papers  (2025-10-07T14:06:30Z)
• Real-time frequency estimation of a qubit without single-shot-readout [1.1796902300802676]
  We propose an adaptive algorithm that controls the readout phase and, therefore, the measurement basis set. We show by numerical simulation that adding the adaptive protocol can further improve the accuracy in a future real-time experiment.
  arXiv  Detail & Related papers  (2022-10-11T15:35:16Z)
• Taming Overconfident Prediction on Unlabeled Data from Hindsight [50.9088560433925]
  Minimizing prediction uncertainty on unlabeled data is a key factor to achieve good performance in semi-supervised learning. This paper proposes a dual mechanism, named ADaptive Sharpening (ADS), which first applies a soft-threshold to adaptively mask out determinate and negligible predictions. ADS significantly improves the state-of-the-art SSL methods by making it a plug-in.
  arXiv  Detail & Related papers  (2021-12-15T15:17:02Z)
• Hessian-Aware Pruning and Optimal Neural Implant [74.3282611517773]
  Pruning is an effective method to reduce the memory footprint and FLOPs associated with neural network models. We introduce a new Hessian Aware Pruning method coupled with a Neural Implant approach that uses second-order sensitivity as a metric for structured pruning.
  arXiv  Detail & Related papers  (2021-01-22T04:08:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.