Beyond Magnitude and Shape: A Direction-Aware Loss for Time Series Forecasting

• URL: http://arxiv.org/abs/2608.01857v1
• Date: Mon, 03 Aug 2026 08:06:39 GMT
• Title: Beyond Magnitude and Shape: A Direction-Aware Loss for Time Series Forecasting
• Abstract summary: We find that MSE-trained forecasters fail on the direction of small moves.<n>We propose CosDir, a simple yet effective direction-aware loss that aligns the difference vectors of the prediction and the target.<n>CosDir is a lightweight, plug-in term that attaches to any backbone without architectural modification.
• Score: 13.359564942661251
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The direction of change --- whether a series will move up or down --- is often as important as its exact value in decisiondriven applications such as risk management and financial forecasting. However, most forecasting losses optimize either point magnitude or shape and frequency structure, and none explicitly targets the direction of change. In this paper, we find that MSE-trained forecasters fail on the direction of small moves. To address this, we propose CosDir, a simple yet effective direction-aware loss that aligns the difference vectors of the prediction and the target via cosine similarity. Being scale-invariant, CosDir keeps a directional gradient on small moves, re-injecting learning signal exactly where MSE neglects it. CosDir is a lightweight, plug-in term that attaches to any backbone without architectural modification. Since the best ratio for mixing the directional and magnitude terms differs across datasets, we further propose CosDir-UW, an extension that makes this ratio adaptive by learning it during training, matching a per-dataset tuned weight with no hyperparameter. We conduct over 100K experiments, demonstrating that our method consistently and significantly improves directional accuracy while preserving magnitude accuracy, and that it outperforms various loss functions. Code is available at: https://github.com/seunghan96/cosdir.

Related papers

• Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors [110.10775872372047]
  We propose emphMagnitude-Direction-Direction (MD) Decoupling.<n>It factorizes each weight into a fixed-norm direction on a hypersphere and learnable per-row and per-column magnitude gains, updated at separate learning rates.<n>Across both Adam and Muon, MD Decoupling improves on well-tuned baselines, transfers the optimal LR across model width without retuning, and continues to help at scale on large Mixture-of-Experts models.
  arXiv  Detail & Related papers  (2026-06-24T15:40:26Z)
• Directional Consistency as a Complementary Optimization Signal: The GONO Framework [0.0]
  We formalize an underexplored phenomenon in deep learning optimization: directional alignment and loss convergence can be decoupled.<n>We introduce GONO, which adapts Adam's momentum beta based on cc_t: amplifying momentum under directional consistency and suppressing it during oscillation.<n>We prove GONO matches Adam's convergence rate and reduces exactly to Adam when the signal is uninformative.
  arXiv  Detail & Related papers  (2026-05-07T17:05:05Z)
• Gravity-aligned Rotation Averaging with Circular Regression [53.81374943525774]
  We introduce a principled approach that integrates gravity direction into the rotation averaging phase of global pipelines. We achieve state-of-the-art accuracy on four large-scale datasets.
  arXiv  Detail & Related papers  (2024-10-16T17:37:43Z)
• Unified Gradient-Based Machine Unlearning with Remain Geometry Enhancement [29.675650285351768]
  Machine unlearning (MU) has emerged to enhance the privacy and trustworthiness of deep neural networks. Approximate MU is a practical method for large-scale models. We propose a fast-slow parameter update strategy to implicitly approximate the up-to-date salient unlearning direction.
  arXiv  Detail & Related papers  (2024-09-29T15:17:33Z)
• Stabilizing Backpropagation Through Time to Learn Complex Physics [21.850601375335074]
  In physics simulations, backpropagating feedback is crucial to acquiring temporally coherent behavior. The alternative vector field we propose follows from two principles: physics simulators have a balanced gradient flow, and certain modifications to the backpropagation pass leave the positions of the original minima unchanged. Our final procedure is easily implementable via a sequence of gradient stopping and component-wise comparison operations.
  arXiv  Detail & Related papers  (2024-05-03T12:20:08Z)
• Step-size Optimization for Continual Learning [5.834516080130717]
  In continual learning, a learner has to keep learning from the data over its whole life time. In a neural network, this can be implemented by using a step-size vector to scale how much samples change network weights. Common algorithms, like RMSProp and Adam, use gradients, specifically normalization, to adapt this step-size vector.
  arXiv  Detail & Related papers  (2024-01-30T19:35:43Z)
• The KFIoU Loss for Rotated Object Detection [115.334070064346]
  In this paper, we argue that one effective alternative is to devise an approximate loss who can achieve trend-level alignment with SkewIoU loss. Specifically, we model the objects as Gaussian distribution and adopt Kalman filter to inherently mimic the mechanism of SkewIoU. The resulting new loss called KFIoU is easier to implement and works better compared with exact SkewIoU.
  arXiv  Detail & Related papers  (2022-01-29T10:54:57Z)
• Correcting Momentum with Second-order Information [50.992629498861724]
  We develop a new algorithm for non-critical optimization that finds an $O(epsilon)$epsilon point in the optimal product. We validate our results on a variety of large-scale deep learning benchmarks and architectures.
  arXiv  Detail & Related papers  (2021-03-04T19:01:20Z)
• Deriving Differential Target Propagation from Iterating Approximate Inverses [91.3755431537592]
  We show that a particular form of target propagation, relying on learned inverses of each layer, which is differential, gives rise to an update rule which corresponds to an approximate Gauss-Newton gradient-based optimization. We consider several iterative calculations based on local auto-encoders at each layer in order to achieve more precise inversions for more accurate target propagation.
  arXiv  Detail & Related papers  (2020-07-29T22:34:45Z)
• AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights [53.8489656709356]
  Normalization techniques are a boon for modern deep learning. It is often overlooked, however, that the additional introduction of momentum results in a rapid reduction in effective step sizes for scale-invariant weights. In this paper, we verify that the widely-adopted combination of the two ingredients lead to the premature decay of effective step sizes and sub-optimal model performances.
  arXiv  Detail & Related papers  (2020-06-15T08:35:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.