Support Topology and Gradient Mixing in Sinkhorn Layers

• URL: http://arxiv.org/abs/2609.07954v1
• Date: Mon, 07 Sep 2026 20:18:56 GMT
• Title: Support Topology and Gradient Mixing in Sinkhorn Layers
• Abstract summary: We show that each row-column cycle induces a row-stochastic operator on column-potential perturbations modulo constants.<n>We derive the accompanying score and marginal source terms and use Dobrushin contraction and minorization to bound homogeneous and source-driven tail cotangents.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sparse Sinkhorn layers use a fixed support graph to restrict transport between tokens. How does this graph control gradient propagation through the scaling iterations. We develop a fixed-support calculus showing that each row-column cycle induces a row-stochastic operator on column-potential perturbations modulo constants. Its transpose propagates zero-mass reverse-mode cotangents. The finite-cycle operator uses two distinct half-step transport plans; at a balanced fixed point it reduces to a two-step walk determined by a single plan. We derive the accompanying score and marginal source terms and use Dobrushin contraction and minorization to bound homogeneous and source-driven tail cotangents. Our main result characterizes when support and marginals guarantee one-step contraction uniformly over finite scores: every feasible face of the transportation polytope must have pairwise two-hop column overlap. Otherwise, suitable score directions make the contraction coefficient arbitrarily close to one. We extend this analysis to ordered support schedules and derive certificates for partition heat-bath layers, coordinate sweeps, forced shared mass, and register-augmented supports. These results provide mathematical criteria for support design in differentiable transport layers, with guarantees restricted to the fixed-support quotient-gradient component.

Related papers

• CyclOT: Learning Quadratic Optimal Transport Maps via Synchronized Forward-Backward Interpolants [20.064386074670605]
  We introduce a bidirectional neural framework in which the learned maps induce forward and backward displacement interpolants.<n>Experiments on Swiss roll, MNIST, CelebA, single-cell data, and chest X-ray images evaluate endpoint fidelity, transport cost inverse, consistency, and the geometry of the induced inverts.
  arXiv  Detail & Related papers  (2026-09-12T11:31:53Z)
• Nonlocality-induced critical-length hierarchy from non-Hermitian competition [48.9823527611878]
  In short-range systems formed by two coupled chains, competition leads to the critical-length law $N_csimln D$.<n>Here we show that long-range hoppings fundamentally reorganizes this critical behavior, producing a hierarchy of distinct scaling laws.<n>Our results offer a platform-independent framework testable in programmable topoelectrical circuits, photonic lattices and digital quantum simulators.
  arXiv  Detail & Related papers  (2026-08-03T18:00:18Z)
• CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning [47.406985340436925]
  CoCurve is a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly.<n>It reduces to one budgeted quadratic program, solved in a single shot under a shared attention--FFN budget, with no labels, fine-tuning, or recovery.
  arXiv  Detail & Related papers  (2026-07-20T05:27:57Z)
• Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Data [3.7437735326653883]
  This paper studies the rate-controlled middle case for full-batch linear classification on separable data.<n>For memoryless stability-annealed smoothed-sign descent with weighted exponential loss, we prove that the normalized iterates converge to the minimizer of a convex Burg-type barrier over a margin slice.
  arXiv  Detail & Related papers  (2026-07-07T08:55:35Z)
• Sector-dominant graph-local drivers for path-window barrier Hamiltonians on the Boolean hypercube [0.0]
  We study finite-size adiabatic state preparation on Boolean hypercubes.<n>We use graph-local drivers built from sector/path coordinates related to monotone Gray-code representatives.<n>We provide code, finite certificates, CSV files, validation logs, and reproduction scripts to make the finite-size claims traceable.
  arXiv  Detail & Related papers  (2026-04-28T10:54:42Z)
• Neural Networks as Local-to-Global Computations [0.0]
  We construct a sheaf from any ReLU neural network by encoding each computational step as a restriction map on an edge.<n>Unlike the forward pass, the heat equation propagates information bidirectionally across layers, enabling pinned neurons that impose constraints in both directions.<n>We validate the framework experimentally on small synthetic tasks, confirming the convergence theorems and demonstrating that sheaf-based training, while not yet competitive with gradient descent, obeys quantitative scaling laws predicted by the theory.
  arXiv  Detail & Related papers  (2026-03-16T05:23:31Z)
• Approaching the scaling limit of transport through lattices with dephasing [0.0]
  We study the stationary--state equations for lattices with generalized Markovian dephasing and relaxation.<n>We present an efficient solution that helps to achieve the scaling limit.<n>There is a wide range of problems that have Markovian relaxation, noise, and driving.
  arXiv  Detail & Related papers  (2025-10-05T06:51:40Z)
• Clustered Switchback Designs for Experimentation Under Spatio-temporal Interference [44.644520116360106]
  We estimate the global average treatment effect (GATE), the difference between average outcomes having exposed all units at all times to treatment or to control.<n>We propose a clustered switchback design, where units are grouped into clusters and time steps are grouped into blocks.<n>We show that for graphs that admit good clustering, a truncated Horvitz-Thompson estimator achieves a $tilde O(1/NT)$ mean squared error (MSE)<n>Our results simultaneously generalize the results from citethu2022switchback,ugander2013graph and citetleung2022rate
  arXiv  Detail & Related papers  (2023-12-25T01:00:58Z)
• Cycle Consistent Probability Divergences Across Different Spaces [38.43511529063335]
  Discrepancy measures between probability distributions are at the core of statistical inference and machine learning. This work proposes a novel unbalanced Monge optimal transport formulation for matching, up to isometries, distributions on different spaces.
  arXiv  Detail & Related papers  (2021-11-22T16:35:58Z)
• Lifting the Convex Conjugate in Lagrangian Relaxations: A Tractable Approach for Continuous Markov Random Fields [53.31927549039624]
  We show that a piecewise discretization preserves better contrast from existing discretization problems. We apply this theory to the problem of matching two images.
  arXiv  Detail & Related papers  (2021-07-13T12:31:06Z)
• Balanced Coarsening for Multilevel Hypergraph Partitioning via Wasserstein Discrepancy [40.93626211612559]
  We propose a balanced coarsening scheme for multilevel hypergraph. An initial partitioning algorithm is designed to improve the quality of k-way hypergraph.
  arXiv  Detail & Related papers  (2021-06-14T15:30:34Z)
• Continuous Regularized Wasserstein Barycenters [51.620781112674024]
  We introduce a new dual formulation for the regularized Wasserstein barycenter problem. We establish strong duality and use the corresponding primal-dual relationship to parametrize the barycenter implicitly using the dual potentials of regularized transport problems.
  arXiv  Detail & Related papers  (2020-08-28T08:28:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.