Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation

• URL: http://arxiv.org/abs/2606.18790v1
• Date: Wed, 17 Jun 2026 08:04:13 GMT
• Title: Closing the Loop: PID Feedback Control for Interpretable Activation Steering in Symbolic Music Generation
• Abstract summary: This paper investigates the mechanistic interpretability of the Multitrack Music Transformer (MMT)<n>We propose a framework for deterministic attribute modulation without retraining to bridge this gap via inference-time activation steering.
• Score: 6.897804369133787
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transformer-based architectures have significantly advanced the generation of complex symbolic sequences, yet a significant gap remains in achieving fine-grained, interpretable control over discrete signal attributes. This paper investigates the mechanistic interpretability of the Multitrack Music Transformer (MMT) and proposes a framework for deterministic attribute modulation without retraining to bridge this gap via inference-time activation steering. Utilizing the Difference-in-Means (DiffMean) methodology, we isolate latent directions for signal attributes, specifically Pitch and Duration, within the residual stream. We validate the Linear Representation Hypothesis in this domain, achieving high correlation between steering magnitude and attribute shift. To address the inherent feature entanglement in multi-attribute steering, we introduce a Dual Steering framework utilizing Gram-Schmidt Orthogonalization. Experimental results demonstrate that this geometric decoupling reduces conceptual interference and signal degradation compared to naive vector addition, enabling independent deterministic control even against strong autoregressive conditioning.

Related papers

• Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation [6.897804369133787]
  This paper investigates the mechanistic interpretability of the Multitrack Music Transformer (MMT)<n>We propose a framework for deterministic attribute modulation without retraining to bridge this gap via inference-time activation steering.
  arXiv  Detail & Related papers  (2026-05-29T13:31:56Z)
• Neural Autoregressive Control Variates for the Quantum Monte Carlo Sign Problem [2.3717692810269586]
  We train a pair of autoregressive models to mitigate the sign problem in quantum Monte Carlo simulations.<n>The two autoregressive networks are confined to the positive- and negative-sign sectors with strictly disjoint support.<n>Their difference is structurally zero-mean, providing an auxiliary observable whose correlation with the sign estimator controls the variance reduction.
  arXiv  Detail & Related papers  (2026-05-26T10:30:44Z)
• Structural Instability of Feature Composition [2.5754366051855837]
  We study the compositional-collapse threshold under a spherical dictionary model.<n>We show that, in the high-bias regime, ReLU converts microscopic correlation-induced variance fluctuations into a systematic drift.<n>We validate the predicted scaling trends on structured semantic features extracted from CLEVR.
  arXiv  Detail & Related papers  (2026-04-18T05:53:31Z)
• Quantum Steering and Entanglement in a Tritter: Hierarchy under Loss [0.10874100424278171]
  We study the correlation hierarchy in a tripartite state engineered by mixing a two-mode squeezed vacuum with a coherent state on a tritter.<n>We analyze the entanglement and Einstein-Podolsky-Rosen steering among the output modes.
  arXiv  Detail & Related papers  (2026-02-08T03:03:16Z)
• Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics [81.80010043113445]
  Local weight fine-tuning, LoRA-based adaptation, and activation-based interventions are studied in isolation.<n>We present a unified view that frames these interventions as dynamic weight updates induced by a control signal.<n>Across methods, we observe a consistent trade-off between preference and utility: stronger control increases preference while predictably reducing utility.
  arXiv  Detail & Related papers  (2026-02-02T17:04:36Z)
• Continuous-Depth Transformers with Learned Control Dynamics [0.0]
  We present a hybrid transformer architecture that replaces discrete middle layers with a continuous-depth Neural Ordinary Differential Equation block.<n>We show that our approach treats depth as a continuous variable governed by a learned vector field $F_(H,, u)$, where $u$ is a low-dimensional control signal injected via explicit concatenation.<n>Our results demonstrate that continuous-depth dynamics with learned control signals provide a viable, efficient mechanism for steerable language generation.
  arXiv  Detail & Related papers  (2026-01-15T02:35:37Z)
• Weights to Code: Extracting Interpretable Algorithms from the Discrete Transformer [65.38883376379812]
  We propose the Discrete Transformer, an architecture engineered to bridge the gap between continuous representations and discrete symbolic logic.<n> Empirically, the Discrete Transformer not only achieves performance comparable to RNN-based baselines but crucially extends interpretability to continuous variable domains.
  arXiv  Detail & Related papers  (2026-01-09T12:49:41Z)
• Deep Delta Learning [91.75868893250662]
  We introduce Deep Delta Learning (DDL), a novel architecture that generalizes the standard residual connection.<n>We provide a spectral analysis of this operator, demonstrating that the gate $(mathbfX)$ enables dynamic between identity mapping, projection, and geometric reflection.<n>This unification empowers the network to explicitly control the spectrum of its layer-wise transition operator, enabling the modeling of complex, non-monotonic dynamics.
  arXiv  Detail & Related papers  (2026-01-01T18:11:38Z)
• Verifying Closed-Loop Contractivity of Learning-Based Controllers via Partitioning [52.23804865017831]
  We address the problem of verifying closed-loop contraction in nonlinear control systems whose controller and contraction metric are both parameterized by neural networks.<n>We derive a tractable and scalable sufficient condition for closed-loop contractivity that reduces to checking that the dominant eigenvalue of a symmetric Metzler matrix is nonpositive.
  arXiv  Detail & Related papers  (2025-12-01T23:06:56Z)
• ERIS: An Energy-Guided Feature Disentanglement Framework for Out-of-Distribution Time Series Classification [51.07970070817353]
  An ideal time series classification (TSC) should be able to capture invariant representations.<n>Current methods are largely unguided, lacking the semantic direction required to isolate truly universal features.<n>We propose an end-to-end Energy-Regularized Information for Shift-Robustness framework to enable guided and reliable feature disentanglement.
  arXiv  Detail & Related papers  (2025-08-19T12:13:41Z)
• Is Disentanglement enough? On Latent Representations for Controllable Music Generation [78.8942067357231]
  In the absence of a strong generative decoder, disentanglement does not necessarily imply controllability. The structure of the latent space with respect to the VAE-decoder plays an important role in boosting the ability of a generative model to manipulate different attributes.
  arXiv  Detail & Related papers  (2021-08-01T18:37:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.