Not All Variables Agree: Reliability-Aware Variable-Wise Gradient Surgery for Multivariate Time-Series Forecasting

• URL: http://arxiv.org/abs/2609.08554v1
• Date: Tue, 08 Sep 2026 10:38:13 GMT
• Title: Not All Variables Agree: Reliability-Aware Variable-Wise Gradient Surgery for Multivariate Time-Series Forecasting
• Abstract summary: We measure the variable-wise gradients directly and find that 30.6% of their pairwise similarities are negative on average across seven datasets.<n>Under shared training 35 of the 64 variables do worse than a full-input single-target oracle.<n>We propose Per-Variable Surgery (PV-Surgery) for backbones with cache-compatible layers.
• Score: 9.878279973302135
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In data-driven training, multivariate time-series forecasting is usually optimized with a scalar loss averaged over samples, variables, and horizons. This averaging is convenient, but the optimizer sees only the aggregated gradient, which does not reveal whether the variable-wise contributions align or oppose one another. To quantify how often this disagreement arises, we measure the variable-wise gradients directly and find that 30.6% of their pairwise cosine similarities are negative on average across seven datasets. However, conflict and harm are not the same thing. Under shared training 35 of the 64 variables do worse than a full-input single-target oracle, and the harmed fraction is not reliably predicted by how often gradients conflict. We propose Per-Variable Surgery (PV-Surgery), an optimizer-side training strategy for backbones with cache-compatible layers. One backward pass builds variable-wise gradient proxies from output-side signals and keeps the pointwise forecasting loss. Reliability-aware selection targets layers whose proxy sums closely approximate their shared-gradient slices. Conditional pooling forms anchor and conflict pools without dropping variables. Common-direction surgery aligns variable or pooled gradients with their normalized mean and restores input norms to avoid reweighting. In experiments across five backbones, seven datasets, and four horizons, PV-Surgery lowers MSE by 3.61% and MAE by 2.93% on average. For multivariate forecasting, this indicates that the variable-wise structure hidden by mean-loss training is a usable optimization signal.

Related papers

• On-Policy Self-Distillation in Diffusion Models [89.32656931795293]
  Reinforcement learning can align diffusion models with human preferences and task-specific objectives, but endpoint rewards do not specify how an intermediate denoising prediction should change.<n>We introduce DiffusionOPSD as an on-policy self-distillation framework that converts image-level reward guidance into explicit targets for clean-output predictions at sampled queries.<n>Across SD 3.5-M and the step-distilled Z-Image-Turbo, our approach achieves the best final held-out scores in 19 of 20 reward-matched settings across two backbones and ten evaluators.
  arXiv  Detail & Related papers  (2026-08-25T14:55:24Z)
• Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training [51.18425726646089]
  We propose textscSURE, a unified latent-space framework for image and video diffusion models.<n>It learns reward distributions and directly uses their reliability to guide dense post-training.<n>textscSURE-REFL achieves the highest VBench quality, semantic, and total scores among the evaluated methods.
  arXiv  Detail & Related papers  (2026-08-06T14:55:42Z)
• The Loss Does Not See the Basis, but Adam Does [0.0]
  We show that gradient descent on a factored model $W = UVtop$ is implicitly biased toward low-rank solutions.<n>We then sort nine update rules on underdetermined matrix sensing by recovery error against the planted ground truth.
  arXiv  Detail & Related papers  (2026-08-05T17:56:26Z)
• A Tale of Two Variances: When Single-Seed Benchmarks Fail in Bayesian Deep Learning [10.891872788947298]
  Local CRPS variance provides a direct signal of single-seed estimation error.<n>Power-law fit quality and monotonicity together provide compact method-level summaries of trajectory regularity.
  arXiv  Detail & Related papers  (2026-04-25T02:52:33Z)
• Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression [102.24287051757469]
  We study self-supervised covariance estimation in deep heteroscedastic regression.<n>We derive an upper bound on the 2-Wasserstein distance between normal distributions.<n>Experiments over a wide range of synthetic and real datasets demonstrate that the proposed 2-Wasserstein bound coupled with pseudo label annotations results in a computationally cheaper yet accurate deep heteroscedastic regression.
  arXiv  Detail & Related papers  (2025-02-14T22:37:11Z)
• Adapting to Latent Subgroup Shifts via Concepts and Proxies [82.01141290360562]
  We show that the optimal target predictor can be non-parametrically identified with the help of concept and proxy variables available only in the source domain. For continuous observations, we propose a latent variable model specific to the data generation process at hand.
  arXiv  Detail & Related papers  (2022-12-21T18:30:22Z)
• Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learning [99.14132861655223]
  We propose a novel intra-class adaptive augmentation (IAA) framework for deep metric learning. We reasonably estimate intra-class variations for every class and generate adaptive synthetic samples to support hard samples mining. Our method significantly improves and outperforms the state-of-the-art methods on retrieval performances by 3%-6%.
  arXiv  Detail & Related papers  (2022-11-29T14:52:38Z)
• Gradient Estimation for Binary Latent Variables via Gradient Variance Clipping [6.234350105794441]
  gradient estimation is often necessary for fitting generative models with discrete latent variables. DisARM and other estimators have potentially exploding variance near the boundary of the parameter space. We propose a new gradient estimator textitbitflip-1 that has lower variance at the boundaries of the parameter space.
  arXiv  Detail & Related papers  (2022-08-12T05:37:52Z)
• Distances with mixed type variables some modified Gower's coefficients [0.0]
  The choice of the distance function depends mainly on the type of the selected variables. The most popular distance for mixed type variables is derived as the complement of the Gower's similarity coefficient. This article tries to address the main drawbacks that affect the overall unweighted Gower's distance.
  arXiv  Detail & Related papers  (2021-01-07T11:00:57Z)
• Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift [81.74795324629712]
  We call prediction-time batch normalization, which significantly improves model accuracy and calibration under covariate shift. We show that prediction-time batch normalization provides complementary benefits to existing state-of-the-art approaches for improving robustness. The method has mixed results when used alongside pre-training, and does not seem to perform as well under more natural types of dataset shift.
  arXiv  Detail & Related papers  (2020-06-19T05:08:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.