Task-Restricted Symmetries in Recurrent Weight Space

• URL: http://arxiv.org/abs/2606.18457v1
• Date: Tue, 16 Jun 2026 20:04:07 GMT
• Title: Task-Restricted Symmetries in Recurrent Weight Space
• Abstract summary: Recurrent networks can contain substantial functional redundancy in weight space.<n>We study this redundancy in one-layer tanh RNNs using ordered real Schur coordinates.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recurrent networks can contain substantial functional redundancy in weight space: changing a recurrent matrix may leave the input-output rollout nearly unchanged on a task distribution, while similar-scale changes can destroy the same behavior. We study this redundancy in one-layer tanh RNNs using ordered real Schur coordinates. The Schur form separates spectral blocks from directed nonnormal couplings, giving a diagnostic basis for structured ablations that keep the input and readout maps fixed. In a fixed-length copy task, selected nonnormal Schur couplings can be removed with little loss in some trained solutions, whereas other couplings are necessary for accurate autonomous replay. Across flip-flop, sine generation, and context-dependent integration, the loss-preserving ablation profile varies across tasks and trained solutions. These results identify candidate approximate functional invariances, not universal symmetries of recurrent weight space. Schur-coordinate ablations provide a practical diagnostic for which structured perturbations preserve a trained recurrent solution and which ones disrupt its computation.

Related papers

• Beyond Legal Spacing: A Residual-Aware Characterization of Entangling-Zone Spacing in Neutral-Atom Compilation [8.092675025427058]
  Neutral-atom processors rely on spatially arranged qubit arrays and parallel Rydberg entangling gates for scalable execution.<n>This paper studies that gap between geometric legality and residual noise by treating entangling-zone spacing as a cross-layer reliability-parallelism variable anchored to experimental neutral-atom geometry.
  arXiv  Detail & Related papers  (2026-08-18T03:40:59Z)
• The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks [0.0]
  We introduce the quadrilateral loss, a differentiable penalty that treats additivity as a measurable behavior instead.<n>We compare routes to exact additivity, spanning structural masks, behavioral penalties, weight decay, backfitting, the shared-section model, and bagged boosted stumps.
  arXiv  Detail & Related papers  (2026-07-22T14:23:37Z)
• Interference and Retention in Continual Learning [0.0]
  We show that forgetting from learning a new task is exactly the interference energy induced on the old task.<n>IGFA is a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict.
  arXiv  Detail & Related papers  (2026-07-10T08:48:23Z)
• Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation [60.41721951405564]
  Low-Rank Adaptation (LoRA) enables efficient federated fine-tuning of segmentation foundation models for medical imaging.<n>Most federated LoRA methods adopt a uniform aggregation rule, which breaks under the encoder-decoder asymmetry in medical segmentation.<n>This mismatch entangles shared anatomy with site-specific biases and harms generalization.<n>We introduce a Subspace Orthogonality Regularizer that penalizes shared-local collinearity in the effective update space.
  arXiv  Detail & Related papers  (2026-06-07T15:39:28Z)
• Scale redundancy and soft gauge fixing in positively homogeneous neural networks [0.0]
  Neural networks with positively homogeneous activations exhibit an exact continuous reparametrization symmetry.<n>We introduce gauge-adapted coordinates that separate invariant and scale-imbalance directions.<n>Inspired by gauge fixing in field theory, we introduce a soft orbit-selection functional acting only on redundant scale coordinates.
  arXiv  Detail & Related papers  (2026-02-16T13:21:49Z)
• MirrorLA: Reflecting Feature Map for Vision Linear Attention [49.41670925034762]
  Linear attention significantly reduces the computational complexity of Transformers from quadratic to linear, yet it consistently lags behind softmax-based attention in performance.<n>We propose MirrorLA, a geometric framework that substitutes passive truncation with active reorientation.<n>MirrorLA achieves state-of-the-art performance across standard benchmarks, demonstrating that strictly linear efficiency can be achieved without compromising representational fidelity.
  arXiv  Detail & Related papers  (2026-02-04T09:14:09Z)
• Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data [36.287199718605]
  Equivariant neural networks exploit underlying task symmetries to improve generalization.<n>We propose Recurrent Equivariant Constraint Modulation (RECM), a layer-wise constraint modulation mechanism.<n>RECM learns appropriate relaxation levels solely from the training signal and the symmetry properties of each layer's input-target distribution.
  arXiv  Detail & Related papers  (2026-02-02T21:59:35Z)
• ResAD++: Towards Class Agnostic Anomaly Detection via Residual Feature Learning [52.11294707895649]
  This paper explores the problem of class-agnostic anomaly detection (AD)<n>The objective is to train one class-agnostic AD model that can generalize to detect anomalies in diverse new classes from different domains without any retraining or fine-tuning on the target data.<n> Comprehensive experiments on eight real-world AD datasets demonstrate that our ResAD++ can achieve remarkable AD results when directly used in new classes.
  arXiv  Detail & Related papers  (2025-09-28T08:41:05Z)
• Spatial regularisation for improved accuracy and interpretability in keypoint-based registration [5.286949071316761]
  Recent approaches based on unsupervised keypoint detection stand out as very promising for interpretability.<n>Here, we propose a three-fold loss to regularise the spatial distribution of the features.<n>Our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks.
  arXiv  Detail & Related papers  (2025-03-06T14:48:25Z)
• Weight transport through spike timing for robust local gradients [0.5236468296934584]
  plasticity in functional neural networks is frequently expressed as gradient descent on a cost.<n>This imposes symmetry constraints that are difficult to reconcile with local computation.<n>We introduce spike-based alignment learning, which uses spike timing statistics to extract and correct the asymmetry between effective reciprocal connections.
  arXiv  Detail & Related papers  (2025-03-04T14:05:39Z)
• Overcoming Distribution Mismatch in Quantizing Image Super-Resolution Networks [53.23803932357899]
  quantization leads to accuracy loss in image super-resolution (SR) networks. Existing works address this distribution mismatch problem by dynamically adapting quantization ranges during test time. We propose a new quantization-aware training scheme that effectively Overcomes the Distribution Mismatch problem in SR networks.
  arXiv  Detail & Related papers  (2023-07-25T08:50:01Z)
• Adaptive feature recombination and recalibration for semantic segmentation with Fully Convolutional Networks [57.64866581615309]
  We propose recombination of features and a spatially adaptive recalibration block that is adapted for semantic segmentation with Fully Convolutional Networks. Results indicate that Recombination and Recalibration improve the results of a competitive baseline, and generalize across three different problems.
  arXiv  Detail & Related papers  (2020-06-19T15:45:03Z)
• Spatially Adaptive Inference with Stochastic Feature Sampling and Interpolation [72.40827239394565]
  We propose to compute features only at sparsely sampled locations. We then densely reconstruct the feature map with an efficient procedure. The presented network is experimentally shown to save substantial computation while maintaining accuracy over a variety of computer vision tasks.
  arXiv  Detail & Related papers  (2020-03-19T15:36:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.