Attractive and Repulsive Pattern Control in Sequence Generation

• URL: http://arxiv.org/abs/2606.24911v1
• Date: Fri, 19 Jun 2026 10:20:58 GMT
• Title: Attractive and Repulsive Pattern Control in Sequence Generation
• Abstract summary: This paper introduces signed pattern control for variable-order Markov generation with BP-Regular sampling.<n>A weighted recurrence automaton computes an activation R for a chosen family of target patterns.<n>Negative coupling makes the target patterns costly during sampling; positive coupling rewards the same patterns and turns them into controlled attractors.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Variable-order Markov models preserve local symbolic syntax by adapting context length, but long continuations can enter recurring high-order "tunnels": repeated suffixes, locally periodic passages, or copied fragments longer than the formal Markov order. This paper introduces signed pattern control for variable-order Markov generation with BP-Regular sampling. A weighted recurrence automaton computes an activation R for a chosen family of target patterns, and belief propagation samples exactly from P_beta(x) proportional to P_0(x) exp(beta R(x)). Negative coupling makes the target patterns costly during sampling; positive coupling rewards the same patterns and turns them into controlled attractors. The target family may be mined online from overactive generated material, supplied by a score or style vocabulary, or designed as an experimental probe. The main experiments use the online homeostatic case, choosing patterns that become overactive in the sampling history. On six duration-bearing monophonic sources, including Bach and Telemann material, the negative branch reduces generated 8-gram self-reuse, increases the effective number of generated 8-grams, and increases coverage of training-supported 4-gram contexts while preserving substantial lower-order support. A pitch-sequence replication on five Weimar Jazz Database solos gives the same anti-reuse signature outside Baroque material. The same signed mechanism also provides a positive branch for probing attractor basins, phase transitions, and hysteresis in the underlying variable-order model.

Related papers

• DynImmune-BERT: Dynamic Immune Repertoire Modeling with Neural ODE Driven Continuous Transformers [20.989286585908978]
  DynImmune-BERT is a continuous time repertoire model for patient level immune status prediction.<n>A low rank meta adapters supervises clonotypes while keeping the parameter count independent of the number of observed clones.<n>Results indicate that event aware temporal modeling can complement strong static encoders when longitudinal repertoire structure is available.
  arXiv  Detail & Related papers  (2026-07-19T13:25:05Z)
• Hacking Generative Perplexity: Why Unconditional Text Evaluation Needs Distributional Metrics [49.443264461057645]
  Diffusion and continuous flow-based language models have emerged as the leading non-autoregressive alternatives to language modeling.<n>Progress in both paradigms is overwhelmingly tracked by generative perplexity (gen-PPL): the per-token negative log-likelihood of samples under a frozen autoregressive (AR) scorer.<n>We argue that this metric is unsound. By construction, gen-PPL measures only predictability under the scoring AR, not grammaticality or semantic coherence.
  arXiv  Detail & Related papers  (2026-06-07T02:35:56Z)
• Margin-calibrated Classifier Guidance for Property-driven Synthesis Planning [23.782236689125995]
  We introduce Sequence Completion Ranking (SCR), which employs contrastive argumentation and a margin-based loss to calibrate the classifier.<n>SCR substantially improves multi-step solve rates from $16.8%$ (unguided generator) to $78.4%$ with reaction-type guidance and $95.3%$ with Tanimoto guidance.<n>Our method also effectively closes the long-standing diversity gap between templatefree and template-based methods.
  arXiv  Detail & Related papers  (2026-05-13T07:12:53Z)
• Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO [70.38763678943648]
  Reinforcement learning with verifiable rewards (RLVR) has become a central paradigm for improving reasoning and code generation in large language models.<n>Standard GRPO uses sequence aggregation, while recent work has advocated token aggregation as a better alternative.<n>We show that these two rules induce different optimization biases; token aggregation introduces sign-length coupling, while sequence aggregation implicitly downweights longer responses.
  arXiv  Detail & Related papers  (2026-04-14T09:48:46Z)
• Conditioning Protein Generation via Hopfield Pattern Multiplicity [0.0]
  Protein sequence generation via geometric attention produces plausible family members from small alignments without training.<n>We show that a single scalar parameter, added as a bias to the sampler's attention logits, continuously shifts generation from the full family toward a user-specified subset.<n>Experiments on five Pfam families confirm the monotonic relationship between separation and gap across a fourfold range of geometries.
  arXiv  Detail & Related papers  (2026-03-20T16:38:15Z)
• Text Generation Beyond Discrete Token Sampling [74.06071135207635]
  Mixture of Inputs (MoI) is a training-free method for autoregressive generation.<n>MoI consistently improves performance across multiple models including QwQ-32B, Nemotron-Super-49B, Gemma-3-27B, and DAPO-Qwen-32B.
  arXiv  Detail & Related papers  (2025-05-20T18:41:46Z)
• σ-GPTs: A New Approach to Autoregressive Models [19.84252724050016]
  We show that by simply adding a positional encoding for the output, this order can be modulated on-the-fly per-sample. We evaluate our method across various domains, including language modeling, path-solving, and aircraft vertical rate prediction.
  arXiv  Detail & Related papers  (2024-04-15T08:22:47Z)
• Masked Audio Generation using a Single Non-Autoregressive Transformer [90.11646612273965]
  MAGNeT is a masked generative sequence modeling method that operates directly over several streams of audio tokens. We demonstrate the efficiency of MAGNeT for the task of text-to-music and text-to-audio generation. We shed light on the importance of each of the components comprising MAGNeT, together with pointing to the trade-offs between autoregressive and non-autoregressive modeling.
  arXiv  Detail & Related papers  (2024-01-09T14:29:39Z)
• Mitigating the Learning Bias towards Repetition by Self-Contrastive Training for Open-Ended Generation [92.42032403795879]
  We show that pretrained language models (LMs) such as GPT2 still tend to generate repetitive texts. We attribute their overestimation of token-level repetition probabilities to the learning bias. We find that LMs use longer-range dependencies to predict repetitive tokens than non-repetitive ones, which may be the cause of sentence-level repetition loops.
  arXiv  Detail & Related papers  (2023-07-04T07:53:55Z)
• Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior [53.69310441063162]
  This paper proposes a sequential prior in a discrete latent space which can generate more naturally sounding samples. We evaluate the approach using listening tests, objective metrics of automatic speech recognition (ASR) performance, and measurements of prosody attributes.
  arXiv  Detail & Related papers  (2020-02-06T12:35:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.