Order Independence With Finetuning

• URL: http://arxiv.org/abs/2503.23483v1
• Date: Sun, 30 Mar 2025 15:38:43 GMT
• Title: Order Independence With Finetuning
• Abstract summary: Large language models (LLMs) demonstrate remarkable performance on many NLP tasks, yet often exhibit order dependence.<n>Recent work proposes Set-Based Prompting (SBP) as a way to remove order information from designated token subsets.<n>We introduce a fine-tuning strategy that integrates SBP into the training process, "pulling" these set-formatted prompts closer to the model's training manifold.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) demonstrate remarkable performance on many NLP tasks, yet often exhibit order dependence: simply reordering semantically identical tokens (e.g., answer choices in multiple-choice questions) can lead to inconsistent predictions. Recent work proposes Set-Based Prompting (SBP) as a way to remove order information from designated token subsets, thereby mitigating positional biases. However, applying SBP on base models induces an out-of-distribution input format, which can degrade in-distribution performance. We introduce a fine-tuning strategy that integrates SBP into the training process, "pulling" these set-formatted prompts closer to the model's training manifold. We show that SBP can be incorporated into a model via fine-tuning. Our experiments on in-distribution (MMLU) and out-of-distribution (CSQA, ARC Challenge) multiple-choice tasks show that SBP fine-tuning significantly improves accuracy and robustness to answer-order permutations, all while preserving broader language modeling capabilities. We discuss the broader implications of order-invariant modeling and outline future directions for building fairer, more consistent LLMs.

Related papers

• DPRM: A Plug-in Doob h transform-induced Token-Ordering Module for Diffusion Language Models [76.12556589212666]
  We introduce DPRM (Doob h-transform Process Reward Model), a plug-in token-ordering module for diffusion language models.<n>DPRM starts from confidence-driven progressive ordering and gradually shifts to Doob h transform Process Reward guided ordering through online estimates.<n>Under tractable optimization assumptions, DPRM also yields a sample-complexity advantage over random and confidence-only ordering.
  arXiv  Detail & Related papers  (2026-04-27T11:50:26Z)
• PLR: Plackett-Luce for Reordering In-Context Learning Examples [13.775690509818753]
  In-context learning adapts large language models by conditioning on a small set of ICL examples.<n>PLR replaces discrete ordering search with learning a probability distribution over orderings with the Plackett-Luce model.<n>PLR consistently improves few-shot accuracy for $k in 4, 8, 16, 32$ examples.
  arXiv  Detail & Related papers  (2026-03-22T19:15:59Z)
• Reasoning Palette: Modulating Reasoning via Latent Contextualization for Controllable Exploration for (V)LMs [49.66344956133349]
  Reasoning capacity shapes both inference-time performance and reinforcement learning (RL) training for large (vision-) language models.<n>This paper proposes Reasoning Palette, a novel latent-modulation framework that endows the model with a latent variable for strategic contextualization.
  arXiv  Detail & Related papers  (2025-12-19T03:32:53Z)
• Listwise Preference Diffusion Optimization for User Behavior Trajectories Prediction [41.53271688465831]
  We formulate User Behavior Trajectory Prediction (UBTP) as a new task setting that explicitly models long-term user preferences.<n>We introduce Listwise Preference Diffusion Optimization (LPDO), a diffusion-based training framework that directly optimize structured preferences over entire item sequences.<n>To rigorously evaluate multi-step prediction quality, we propose the task-specific metric Sequential Match (SeqMatch), which measures exact trajectory agreement, and adopt Perplexity (PPL), which assesses probabilistic fidelity.
  arXiv  Detail & Related papers  (2025-11-01T12:16:24Z)
• Probabilistic Token Alignment for Large Language Model Fusion [100.30692772017238]
  Training large language models (LLMs) from scratch can yield models with unique functionalities and strengths, but it is costly and often leads to redundant capabilities.<n>A key challenge in existing model fusion is their dependence on manually predefined vocabulary alignment.<n>We propose the probabilistic token alignment method as a general and soft mapping for alignment, named as PTA-LLM.
  arXiv  Detail & Related papers  (2025-09-21T23:18:24Z)
• Reinforced Context Order Recovery for Adaptive Reasoning and Planning [23.229513376337607]
  Current causal and diffusion models encounter difficulties in problems that require adaptive token generation orders to solve tractably.<n>Motivated by this, we propose Reinforced Context Order Recovery (ReCOR), a reinforcement-learning-based framework to extract adaptive, data-dependent token generation orders.
  arXiv  Detail & Related papers  (2025-08-18T16:42:55Z)
• Set-LLM: A Permutation-Invariant LLM [2.9665130256021]
  This paper is motivated by a specific vulnerability: the order sensitivity of large language models (LLMs)<n>We introduce Set-LLM, a novel architectural adaptation for pretrained LLMs that enables the processing of mixed set-text inputs with permutation invariance guarantees.
  arXiv  Detail & Related papers  (2025-05-21T12:14:26Z)
• Order-Independence Without Fine Tuning [18.020492646988746]
  We present Set-Based Prompting, a technique that guarantees the output of an LLM will not have order dependence on a specified set of sub-sequences.<n>Despite our inputs being out of distribution, the impact on expected accuracy is small, where the expectation is over the order of uniformly chosen shuffling of the candidate responses.
  arXiv  Detail & Related papers  (2024-06-04T16:09:13Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)
• Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference [32.62084449979531]
  We extend SortedNet to generative NLP tasks by replacing Standard Fine-Tuning (SFT) with Sorted Fine-Tuning (SoFT) Our approach boosts model efficiency, eliminating the need for multiple models for various scenarios during inference. Our results show the superior performance of sub-models in comparison to Standard Fine-Tuning and SFT+ICT (Early-Exit)
  arXiv  Detail & Related papers  (2023-09-16T11:58:34Z)
• Large Language Models Are Not Robust Multiple Choice Selectors [117.72712117510953]
  Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs) This work shows that modern LLMs are vulnerable to option position changes due to their inherent "selection bias" We propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution.
  arXiv  Detail & Related papers  (2023-09-07T17:44:56Z)
• Instruction Position Matters in Sequence Generation with Large Language Models [67.87516654892343]
  Large language models (LLMs) are capable of performing conditional sequence generation tasks, such as translation or summarization. We propose enhancing the instruction-following capability of LLMs by shifting the position of task instructions after the input sentences.
  arXiv  Detail & Related papers  (2023-08-23T12:36:57Z)
• Inconsistencies in Masked Language Models [20.320583166619528]
  Masked language models (MLMs) can provide distributions of tokens in the masked positions in a sequence. distributions corresponding to different masking patterns can demonstrate considerable inconsistencies. We propose an inference-time strategy for fors called Ensemble of Conditionals.
  arXiv  Detail & Related papers  (2022-12-30T22:53:25Z)
• Sequence-to-Set Generative Models [9.525560801277903]
  We propose a sequence-to-set method to transform any sequence generative model into a set generative model. We present GRU2Set, which is an instance of our sequence-to-set method and employs the famous GRU model as the sequence generative model. A direct application of our models is to learn an order/set distribution from a collection of e-commerce orders.
  arXiv  Detail & Related papers  (2022-09-19T07:13:51Z)
• Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little [74.49773960145681]
  A possible explanation for the impressive performance of masked language model (MLM)-training is that such models have learned to represent the syntactic structures prevalent in NLP pipelines. In this paper, we propose a different explanation: pre-trains succeed on downstream tasks almost entirely due to their ability to model higher-order word co-occurrence statistics. Our results show that purely distributional information largely explains the success of pre-training, and underscore the importance of curating challenging evaluation datasets that require deeper linguistic knowledge.
  arXiv  Detail & Related papers  (2021-04-14T06:30:36Z)
• Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning [61.32992639292889]
  Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. We introduce a new scoring method that casts a plausibility ranking task in a full-text format. We show that our method provides a much more stable training phase across random restarts.
  arXiv  Detail & Related papers  (2020-04-29T10:54:40Z)
• Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors [75.58555462743585]
  Naively trained neural networks tend to experience catastrophic forgetting in sequential task settings. We propose a principled nonparametric approach based on the Indian Buffet Process (IBP) prior, letting the data determine how much to expand the model complexity. We demonstrate the effectiveness of our method on a number of continual learning benchmarks and analyze how weight factors are allocated and reused throughout the training.
  arXiv  Detail & Related papers  (2020-04-21T15:20:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.