It's All Just Vectorization: einx, a Universal Notation for Tensor Operations

• URL: http://arxiv.org/abs/2607.27987v1
• Date: Thu, 30 Jul 2026 10:33:04 GMT
• Title: It's All Just Vectorization: einx, a Universal Notation for Tensor Operations
• Abstract summary: We introduce einx, a universal notation for tensor operations.<n>It uses declarative, pointful expressions that are defined by analogy with loop notation.<n>einx is embedded in Python and integrates seamlessly with existing tensor frameworks.
• Score: 6.439761523935615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tensor operations represent a cornerstone of modern scientific computing. However, the Numpy-like notation adopted by predominant tensor frameworks is often difficult to read and write and prone to so-called shape errors, i.a., due to following inconsistent rules across a large, complex collection of operations. Alternatives like einsum and einops have gained popularity, but are inherently restricted to few operations and lack the generality required for a universal model of tensor programming. To derive a better paradigm, we revisit vectorization as a function for transforming tensor operations, and use it to both lift lower-order operations to higher-order operations, and conceptually decompose higher-order operations to lower-order operations and their vectorization. Building on the universal nature of vectorization, we introduce einx, a universal notation for tensor operations. It uses declarative, pointful expressions that are defined by analogy with loop notation and represent the vectorization of tensor operations. The notation reduces the large APIs of existing frameworks to a small set of elementary operations, applies consistent rules across all operations, and enables a clean, readable and writable representation in code. We provide an implementation of einx that is embedded in Python and integrates seamlessly with existing tensor frameworks: https://github.com/fferflo/einx

Related papers

• Functional Attention: From Pairwise Affinities to Functional Correspondences [48.36268255718624]
  We introduce emphFunctional Attention, which reinterprets attention as a functional correspondence between adaptive bases.<n>Experiments demonstrate that emphFunctional Attention can match state-of-the-art performance in many operator learning tasks.
  arXiv  Detail & Related papers  (2026-05-29T17:22:19Z)
• SeQuant Framework for Symbolic and Numerical Tensor Algebra. I. Core Capabilities [0.9556128246747769]
  SeQuant is an open-source library for symbolic algebra of tensors over commutative (scalar) and non-commutative (operator) rings.<n>The key innovation supporting most of its functionality is a graph-theoretic tensor network canonicalizer.
  arXiv  Detail & Related papers  (2025-11-13T04:17:05Z)
• Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic [51.41777906371754]
  Large language models often require costly optimization, such as reinforcement learning, to master complex reasoning tasks.<n>This work demonstrates that reasoning ability, once learned, can be extracted and transferred between models as a compact task vector.
  arXiv  Detail & Related papers  (2025-09-01T11:04:51Z)
• EinHops: Einsum Notation for Expressive Homomorphic Operations on RNS-CKKS Tensors [3.0088450191132394]
  Homomorphic Encryption (FHE) allows computation to be performed directly on encrypted data.<n>FHE provides a limited instruction set: SIMD addition, SIMD multiplication, and cyclic rotation of 1-D vectors.<n>We present EinHops, a minimalist system that factors einsum expressions into a fixed sequence of FHE operations.
  arXiv  Detail & Related papers  (2025-07-10T17:50:10Z)
• Overcomplete Tensor Decomposition via Koszul-Young Flattenings [56.82556231289414]
  We give a new algorithm for decomposing an $n_times n times n_3$ tensor as the sum of a minimal number of rank-1 terms.<n>We show that an even more general class of degree-$d$s cannot surpass rank $Cn$ for a constant $C = C(d)$.
  arXiv  Detail & Related papers  (2024-11-21T17:41:09Z)
• An introduction to graphical tensor notation for mechanistic interpretability [0.0]
  It's often easy to get confused about which operations are happening between tensors. The first half of this document introduces the notation and applies it to some decompositions. The second half applies it to some existing some foundational approaches for mechanistically understanding language models.
  arXiv  Detail & Related papers  (2024-02-02T02:56:01Z)
• TensorKrowch: Smooth integration of tensor networks in machine learning [46.0920431279359]
  We introduceKrowch, an open source Python library built on top of PyTorch. Krowch allows users to construct any tensor network, train it, and integrate it as a layer in more intricate deep learning models.
  arXiv  Detail & Related papers  (2023-06-14T15:55:19Z)
• Near-Linear Time and Fixed-Parameter Tractable Algorithms for Tensor Decompositions [51.19236668224547]
  We study low rank approximation of tensors, focusing on the tensor train and Tucker decompositions. For tensor train decomposition, we give a bicriteria $(1 + eps)$-approximation algorithm with a small bicriteria rank and $O(q cdot nnz(A))$ running time. In addition, we extend our algorithm to tensor networks with arbitrary graphs.
  arXiv  Detail & Related papers  (2022-07-15T11:55:09Z)
• A Unified Framework for Implicit Sinkhorn Differentiation [58.56866763433335]
  We propose an algorithm that obtains analytical gradients of a Sinkhorn layer via implicit differentiation. We show that it is computationally more efficient, particularly when resources like GPU memory are scarce.
  arXiv  Detail & Related papers  (2022-05-13T14:45:31Z)
• Stack operation of tensor networks [10.86105335102537]
  We propose a mathematically rigorous definition for the tensor network stack approach. We illustrate the main ideas with the matrix product states based machine learning as an example.
  arXiv  Detail & Related papers  (2022-03-28T12:45:13Z)
• The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Padding [14.635810503599759]
  CoRa is a tensor compiler that allows users to easily generate efficient code for ragged tensor operators. We evaluate CoRa on a variety of operators on ragged tensors as well as on an encoder layer of the transformer model.
  arXiv  Detail & Related papers  (2021-10-19T19:39:04Z)
• Named Tensor Notation [117.30373263410507]
  We propose a notation for tensors with named axes. It relieves the author, reader, and future implementers from the burden of keeping track of the order of axes. It also makes it easy to extend operations on low-order tensors to higher order ones.
  arXiv  Detail & Related papers  (2021-02-25T22:21:30Z)
• OPFython: A Python-Inspired Optimum-Path Forest Classifier [68.8204255655161]
  This paper proposes a Python-based Optimum-Path Forest framework, denoted as OPFython. As OPFython is a Python-based library, it provides a more friendly environment and a faster prototyping workspace than the C language.
  arXiv  Detail & Related papers  (2020-01-28T15:46:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.