Near-Universal Multiplicative Updates for Nonnegative Einsum Factorization

• URL: http://arxiv.org/abs/2602.02759v2
• Date: Mon, 09 Feb 2026 15:21:13 GMT
• Title: Near-Universal Multiplicative Updates for Nonnegative Einsum Factorization
• Authors: John Hood, Aaron Schein,
• Abstract summary: NNEinFact is an einsum-based multiplicative update algorithm that fits any nonnegative tensor factorization expressible as a tensor contraction.<n>It converges to a stationary point of the loss, supports missing data, and fits to tensors with hundreds of millions of entries in seconds.<n>It attains less than half the test loss of gradient-based methods while converging up to 90 times faster.
• Score: 3.81006880749861
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the ubiquity of multiway data across scientific domains, there are few user-friendly tools that fit tailored nonnegative tensor factorizations. Researchers may use gradient-based automatic differentiation (which often struggles in nonnegative settings), choose between a limited set of methods with mature implementations, or implement their own model from scratch. As an alternative, we introduce NNEinFact, an einsum-based multiplicative update algorithm that fits any nonnegative tensor factorization expressible as a tensor contraction by minimizing one of many user-specified loss functions (including the $(α,β)$-divergence). To use NNEinFact, the researcher simply specifies their model with a string. NNEinFact converges to a stationary point of the loss, supports missing data, and fits to tensors with hundreds of millions of entries in seconds. Empirically, NNEinFact fits custom models which outperform standard ones in heldout prediction tasks on real-world tensor data by over $37\%$ and attains less than half the test loss of gradient-based methods while converging up to 90 times faster.

Related papers

• TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training [91.8932638236073]
  We introduce textbfTensorGRaD, a novel method that directly addresses the memory challenges associated with large-structured weights.<n>We show that sparseGRaD reduces total memory usage by over $50%$ while maintaining and sometimes even improving accuracy.
  arXiv  Detail & Related papers  (2025-01-04T20:51:51Z)
• Multi-Dictionary Tensor Decomposition [5.733331864416094]
  We propose a framework for Multi-Dictionary Decomposition (MDTD) We derive a general optimization algorithm for MDTD that handles both complete input and input with missing values. It can impute missing values in billion-entry tensors more accurately and scalably than state-of-the-art competitors.
  arXiv  Detail & Related papers  (2023-09-18T12:31:56Z)
• A Novel Tensor Factorization-Based Method with Robustness to Inaccurate Rank Estimation [9.058215418134209]
  We propose a new tensor norm with a dual low-rank constraint, which utilizes the low-rank prior and rank information at the same time. It is proven theoretically that the resulting tensor completion model can effectively avoid performance degradation caused by inaccurate rank estimation. Based on this, the total cost at each iteration of the optimization algorithm is reduced to $mathcalO(n3log n +kn3)$ from $mathcalO(n4)$ achieved with standard methods.
  arXiv  Detail & Related papers  (2023-05-19T06:26:18Z)
• Truncated tensor Schatten p-norm based approach for spatiotemporal traffic data imputation with complicated missing patterns [77.34726150561087]
  We introduce four complicated missing patterns, including missing and three fiber-like missing cases according to the mode-drivenn fibers. Despite nonity of the objective function in our model, we derive the optimal solutions by integrating alternating data-mputation method of multipliers.
  arXiv  Detail & Related papers  (2022-05-19T08:37:56Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• Augmented Tensor Decomposition with Stochastic Optimization [46.16865811396394]
  Real-world tensor data are usually high-ordered and have large dimensions with millions or billions of entries. It is expensive to decompose the whole tensor with traditional algorithms. This paper proposes augmented tensor decomposition, which effectively incorporates data augmentations to boost downstream classification.
  arXiv  Detail & Related papers  (2021-06-15T06:29:05Z)
• MTC: Multiresolution Tensor Completion from Partial and Coarse Observations [49.931849672492305]
  Existing completion formulation mostly relies on partial observations from a single tensor. We propose an efficient Multi-resolution Completion model (MTC) to solve the problem.
  arXiv  Detail & Related papers  (2021-06-14T02:20:03Z)
• Robust Implicit Networks via Non-Euclidean Contractions [63.91638306025768]
  Implicit neural networks show improved accuracy and significant reduction in memory consumption. They can suffer from ill-posedness and convergence instability. This paper provides a new framework to design well-posed and robust implicit neural networks.
  arXiv  Detail & Related papers  (2021-06-06T18:05:02Z)
• Multi-version Tensor Completion for Time-delayed Spatio-temporal Data [50.762087239885936]
  Real-world-temporal data is often incomplete or inaccurate due to various data loading delays. We propose a low-rank tensor model to predict the updates over time. We obtain up to 27.2% lower root mean-squared-error compared to the best baseline method.
  arXiv  Detail & Related papers  (2021-05-11T19:55:56Z)
• Beyond Lazy Training for Over-parameterized Tensor Decomposition [69.4699995828506]
  We show that gradient descent on over-parametrized objective could go beyond the lazy training regime and utilize certain low-rank structure in the data. Our results show that gradient descent on over-parametrized objective could go beyond the lazy training regime and utilize certain low-rank structure in the data.
  arXiv  Detail & Related papers  (2020-10-22T00:32:12Z)
• A Solution for Large Scale Nonlinear Regression with High Rank and Degree at Constant Memory Complexity via Latent Tensor Reconstruction [0.0]
  This paper proposes a novel method for learning highly nonlinear, multivariate functions from examples. Our method takes advantage of the property that continuous functions can be approximated by bys, which in turn are representable by tensors. For learning the models, we present an efficient-based algorithm that can be implemented in linear time.
  arXiv  Detail & Related papers  (2020-05-04T14:49:14Z)
• Sparse and Low-Rank High-Order Tensor Regression via Parallel Proximal Method [6.381138694845438]
  We propose the Sparse and Low-rank Regression model for large-scale data with high-order structures. Our model enforces sparsity and low-rankness of the tensor coefficient. Our model's predictions exhibit meaningful interpretations on the video dataset.
  arXiv  Detail & Related papers  (2019-11-29T06:25:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.