Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior

• URL: http://arxiv.org/abs/2607.18804v1
• Date: Tue, 21 Jul 2026 07:34:18 GMT
• Title: Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posterior
• Abstract summary: We introduce textbfPosterior Prefix Tuning (PPT), a new method for empheliciting behavior from a transformer.<n>For a BFT, the elicitation objective factors through the latent posterior, and the gradient of this objective can be estimated from samples of the prior alone.
• Score: 4.347699959341557
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In the \emph{latent posterior model} of transformer behavior, the next-token distribution arises from a posterior over latent predictive models conditioned on the context, mixed to generate continuations. We exploit this model in settings where it is exact, namely Bayes-filtered transformers (BFTs) meta-learned on sequences from a hierarchical prior, to introduce \textbf{Posterior Prefix Tuning (PPT)}, a new method for \emph{eliciting} behavior from a transformer: given a utility function on continuations, find a prompt under which the transformer generates continuations of high expected utility. For a BFT, the elicitation objective factors through the latent posterior, and the gradient of this objective can be estimated from samples of the prior alone. PPT optimizes the parameters of a distribution over hard prompts: it draws prior samples once from the BFT via predictive Monte Carlo (PMC), then estimates the gradient by importance sampling against them. The optimization performs no transformer forward passes and no backpropagation through the transformer, and the prior samples are utility-independent, so a single set of samples drives elicitation against any number of utilities at negligible marginal cost. We validate PPT on Beta--Bernoulli and reinforced urn BFTs across three utility families (reverse cross-entropy, frequency matching, Dyck validity).

Related papers

• PriFT: Prior-Support Guided Supervised Fine-Tuning [74.65198014829393]
  Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation.<n>SFT fits fixed demonstrations token by token, including targets poorly aligned with the model's pretrained distribution.<n>We propose PriFT, which derives token weights from a frozen pretrained reference to obtain a stable reweighting signal unaffected by fine-tuning.
  arXiv  Detail & Related papers  (2026-06-08T12:14:06Z)
• N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation [68.47358899451255]
  N-vium is a mixture-of-exits transformer that partially parallelizes computation across depth on standard hardware.<n>N-vium attaches prediction heads at multiple depths and defines the next-token distribution as a learned mixture over these exits.
  arXiv  Detail & Related papers  (2026-05-13T08:46:17Z)
• PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks [7.188084723389871]
  We propose PFN-TS, a Thompson sampling algorithm that converts PFN posterior predictives into mean-reward samples.<n>PFN-TS achieves the best average rank across nonlinear synthetic and OpenML classification-to-bandit benchmarks.
  arXiv  Detail & Related papers  (2026-05-11T07:46:10Z)
• Improving DNS Exfiltration Detection via Transformer Pretraining [0.0]
  In-domain pretraining of Bidirectional Representations from Transformer (BERT) model improves subdomain-level detection of exfiltration at low false positive rates.<n>We develop a controlled pipeline where we freeze operating points on validation and transfer them to the test set.
  arXiv  Detail & Related papers  (2026-04-09T15:58:34Z)
• Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models [102.20309135516186]
  Cross-entropy (CE) training provides dense and scalable supervision for language models.<n>We introduce a feature-matching objective for language-model fine-tuning.<n>To optimize this objective efficiently, we propose energy-based fine-tuning.
  arXiv  Detail & Related papers  (2026-03-12T17:57:50Z)
• FBS: Modeling Native Parallel Reading inside a Transformer [0.0]
  Large language models (LLMs) excel across many tasks, yet inference is still dominated by strictly token-by-token autoregression.<n>We propose the textbfFovea-Block-Skip Transformer (FBS), which injects a causal, trainable loop into Transformers via Parafovea-Attention Window (PAW), Chunk-Head (CH), and Skip-Gate (SG)
  arXiv  Detail & Related papers  (2026-01-29T13:39:55Z)
• Adaptive Pruning of Pretrained Transformer via Differential Inclusions [48.47890215458465]
  Current compression algorithms prune transformers at fixed compression ratios, requiring a unique pruning process for each ratio.<n>We propose pruning of pretrained transformers at any desired ratio within a single pruning stage, based on a differential inclusion for a mask parameter.<n>This dynamic can generate the whole regularization solution path of the mask parameter, whose support set identifies the network structure.
  arXiv  Detail & Related papers  (2025-01-06T06:34:52Z)
• GBT: Two-stage transformer framework for non-stationary time series forecasting [3.830797055092574]
  We propose GBT, a novel two-stage Transformer framework with Good Beginning. It decouples the prediction process of TSFT into two stages, including Auto-Regression stage and Self-Regression stage. Experiments on seven benchmark datasets demonstrate that GBT outperforms SOTA TSFTs with only canonical attention and convolution.
  arXiv  Detail & Related papers  (2023-07-17T07:55:21Z)
• Trained Transformers Learn Linear Models In-Context [39.56636898650966]
  Attention-based neural networks as transformers have demonstrated a remarkable ability to exhibit inattention learning (ICL) We show that when transformer training over random instances of linear regression problems, these models' predictions mimic nonlinear of ordinary squares.
  arXiv  Detail & Related papers  (2023-06-16T15:50:03Z)
• Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost [53.746169882193456]
  Recent works have proposed various sparse attention modules to overcome the quadratic cost of self-attention. We propose a model that resolves both problems by endowing each attention head with a mixed-membership Block Model. Our model outperforms previous efficient variants as well as the original Transformer with full attention.
  arXiv  Detail & Related papers  (2022-10-27T15:30:52Z)
• Sample-Efficient Optimisation with Probabilistic Transformer Surrogates [66.98962321504085]
  This paper investigates the feasibility of employing state-of-the-art probabilistic transformers in Bayesian optimisation. We observe two drawbacks stemming from their training procedure and loss definition, hindering their direct deployment as proxies in black-box optimisation. We introduce two components: 1) a BO-tailored training prior supporting non-uniformly distributed points, and 2) a novel approximate posterior regulariser trading-off accuracy and input sensitivity to filter favourable stationary points for improved predictive performance.
  arXiv  Detail & Related papers  (2022-05-27T11:13:17Z)
• Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer [61.818320703583126]
  Sequential Recommendation characterizes the evolving patterns by modeling item sequences chronologically. Recent developments of transformer inspire the community to design effective sequence encoders. We introduce a new framework for textbfAugmenting textbfSequential textbfRecommendation with textbfPseudo-prior items(ASReP)
  arXiv  Detail & Related papers  (2021-05-02T18:06:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.