State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking

• URL: http://arxiv.org/abs/2608.03425v2
• Date: Sat, 08 Aug 2026 05:50:45 GMT
• Title: State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking
• Abstract summary: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms.<n>In this paper, we show that textbfstate propagation alone is sufficient.<n>We propose the textbf State Propagator (CSP), a minimalistic recurrent architecture that textbfonly propagates hidden states across layers without output projections at intermediate steps.
• Score: 8.15785119761963
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms. However, for a class of deterministic state tracking tasks---such as parity checking, modular counting, and parenthesis matching---attention may be overkill. In this paper, we show that \textbf{state propagation alone is sufficient}. We propose the \textbf{Complex State Propagator (CSP)}, a minimalistic recurrent architecture that \textbf{only propagates hidden states} across layers without output projections at intermediate steps. The state is represented as a complex-valued vector, updated via input-dependent rotations in the complex domain. To enable deep propagation without gradient vanishing or degradation, we introduce a \textbf{block-level skip connection} alongside element-wise complex normalization and SiLU activation at sequence boundaries. Applied with Focal Loss, CSP achieves \textbf{100\% accuracy} with perfect F1 scores across canonical tasks.

Related papers

• A Compositional Theory of Causally Masked Transformers [50.88598486616582]
  We develop a formalization that derives expressivity directly from the model's implemented dynamics.<n>Each attention head updates its own state independently within a layer.<n>A modified form of soft attention supports irreversible, checklist-like state.
  arXiv  Detail & Related papers  (2026-07-29T14:47:19Z)
• OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation [66.98820543497212]
  We propose a hierarchical framework centered on textbfcontact flow (CF), a compact representation consisting of key body trajectories and time-series binary contact signals.<n>We additionally collect the OmniContact dataset, a MoCap-based HOI corpus for humanoid loco-manipulation.<n>Together, they enable robust execution, autonomous failure recovery, and flexible composition of meta-skills for long-horizon tasks.
  arXiv  Detail & Related papers  (2026-06-24T16:28:23Z)
• Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories [62.54876287800644]
  Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric.<n>Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction.<n>We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable.
  arXiv  Detail & Related papers  (2026-06-21T14:25:50Z)
• Interdomain Attention: Beyond Token-Level Key-Value Memory [15.708551010135407]
  We propose Interdomain Attention, which integrates an SSM into an attention module through kernel methods.<n>An attention kernel is approximated by a finite feature map, the resulting key features and values are projected onto a shared set of basis functions.<n>Each query attends to the compressed coefficients through its own feature map, recovering query-conditioned attention over a fixed-size state.
  arXiv  Detail & Related papers  (2026-05-23T01:18:00Z)
• Circuit Complexity of Hierarchical Knowledge Tracing and Implications for Log-Precision Transformers [2.688207424884465]
  Empirically, transformer encoders trained on recursion-invariant trees converge to permutation-invariant shortcuts.<n>These findings motivate structure-aware objectives and iterative mechanisms for prerequisite-sensitive knowledge tracing on deep hierarchies.
  arXiv  Detail & Related papers  (2026-03-25T01:10:42Z)
• Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping [61.459927600301654]
  Multi-condition control is bottlenecked by the conventional concatenate-and-attend'' strategy.<n>Our analysis reveals that much of this cross-modal interaction is spatially or semantically redundant.<n>We propose Position-aligned and Keyword-scoped Attention (PKA), a highly efficient framework designed to eliminate these redundancies.
  arXiv  Detail & Related papers  (2026-02-06T16:39:10Z)
• Spectral-Window Hybrid (SWH) [0.0]
  Scaling sequence modeling to extreme contexts requires balancing computational efficiency with representational expressivity.<n>We propose the textbfSpectral-Window Hybrid (SWH), an architecture that decouples sequence modeling into two textitparallel streams.<n>We demonstrate that SWH matches the perplexity of standard Transformers on short contexts while enabling efficient linear scaling to extended sequences.
  arXiv  Detail & Related papers  (2026-01-04T00:31:36Z)
• Dynamic Subspace Composition: Efficient Adaptation via Contractive Basis Expansion [0.0]
  Mixture of Experts (MoE) models scale capacity but often suffer from representation collapse and gradient instability.<n>We propose Dynamic Subspace Composition (DSC), a framework that approximates context-dependent weights via a state-dependent, sparse expansion of a shared basis bank.
  arXiv  Detail & Related papers  (2025-12-29T13:11:01Z)
• When Bayesian Tensor Completion Meets Multioutput Gaussian Processes: Functional Universality and Rank Learning [53.17227599983122]
  Functional tensor decomposition can analyze multi-dimensional data with real-valued indices.<n>We propose a rank-revealing functional low-rank tensor completion (RR-F) method.<n>We establish the universal approximation property of the model for continuous multi-dimensional signals.
  arXiv  Detail & Related papers  (2025-12-25T03:15:52Z)
• Correlation-Aware Feature Attribution Based Explainable AI [4.457502798302293]
  emphExCIR is a correlation-aware attribution score equipped with a lightweight transfer protocol.<n>textscBlockCIR mitigates double-counting in collinear clusters.<n>emphscalable explainability provides emphcomputationally efficient, emphconsistent, and emphscalable explainability for real-world deployment.
  arXiv  Detail & Related papers  (2025-11-20T15:51:00Z)
• A general optimization framework for mapping local transition-state networks [0.0]
  Local transition-state networks reveal the barrier architecture that explains observed behaviour.<n>We present a general optimization framework that systematically expands local coverage by coupling a multi-objective explorer with a bilayer minimum-mode kernel.
  arXiv  Detail & Related papers  (2025-09-30T13:54:10Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• Coupling Global Context and Local Contents for Weakly-Supervised Semantic Segmentation [54.419401869108846]
  We propose a single-stage WeaklySupervised Semantic (WSSS) model with only the image-level class label supervisions. A flexible context aggregation module is proposed to capture the global object context in different granular spaces. A semantically consistent feature fusion module is proposed in a bottom-up parameter-learnable fashion to aggregate the fine-grained local contents.
  arXiv  Detail & Related papers  (2023-04-18T15:29:23Z)
• Embed to Control Partially Observed Systems: Representation Learning with Provable Sample Efficiency [105.17746223041954]
  Reinforcement learning in partially observed Markov decision processes (POMDPs) faces two challenges. It often takes the full history to predict the future, which induces a sample complexity that scales exponentially with the horizon. We propose a reinforcement learning algorithm named Embed to Control (ETC), which learns the representation at two levels while optimizing the policy.
  arXiv  Detail & Related papers  (2022-05-26T16:34:46Z)
• Combiner: Full Attention Transformer with Sparse Computation Cost [142.10203598824964]
  We propose Combiner, which provides full attention capability in each attention head while maintaining low computation complexity. We show that most sparse attention patterns used in existing sparse transformers are able to inspire the design of such factorization for full attention. An experimental evaluation on both autoregressive and bidirectional sequence tasks demonstrates the effectiveness of this approach.
  arXiv  Detail & Related papers  (2021-07-12T22:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.