When Models Decide and When They Bind: A Two-Stage Computation for Multiple-Choice Question-Answering

• URL: http://arxiv.org/abs/2601.03914v1
• Date: Wed, 07 Jan 2026 13:27:48 GMT
• Title: When Models Decide and When They Bind: A Two-Stage Computation for Multiple-Choice Question-Answering
• Abstract summary: Multiple-choice question answering (MCQA) is easy to evaluate but adds a meta-task.<n>We study how language models implement MCQA internally using representational analyses (PCA, linear probes) as well as causal interventions.
• Score: 7.622274098558385
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multiple-choice question answering (MCQA) is easy to evaluate but adds a meta-task: models must both solve the problem and output the symbol that *represents* the answer, conflating reasoning errors with symbol-binding failures. We study how language models implement MCQA internally using representational analyses (PCA, linear probes) as well as causal interventions. We find that option-boundary (newline) residual states often contain strong linearly decodable signals related to per-option correctness. Winner-identity probing reveals a two-stage progression: the winning *content position* becomes decodable immediately after the final option is processed, while the *output symbol* is represented closer to the answer emission position. Tests under symbol and content permutations support a two-stage mechanism in which models first select a winner in content space and then bind or route that winner to the appropriate symbol to emit.

Related papers

• Auditing Question-Order Effects in Large Language Models with the QQ Equality: Mechanism Characterization and a Saturation Caveat [4.07636450847048]
  Human survey respondents exhibit question-order effects that satisfy the QQ (quantum question) equality.<n>We develop the QQ equality into an audit criterion for sequential judgments of autoregressive large language models.
  arXiv  Detail & Related papers  (2026-07-19T12:22:06Z)
• VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning? [79.44137632338062]
  When a multimodal large language model answers a visual reasoning question correctly, is the prediction actually supported by the task-critical visual evidence?<n>We introduce VisualFLIP, a paired benchmark with 1,374 images arranged as same-question pairs across cardinality, attribute, spatial, and logic tasks.<n>We evaluate 24 MLLMs with pair accuracy, which requires solving both sides of a pair, and Collapse Rate (CR), which measures how often a model that solves at least one side repeats the same non-empty answer for both images.
  arXiv  Detail & Related papers  (2026-06-05T22:06:07Z)
• Reasoning Models are Test Exploiters: Rethinking Multiple-Choice [12.317748510370238]
  Large Language Models (LLMs) are asked to choose among a fixed set of choices.<n>Multiple-choice question-answering (McQCA) is a good proxy for the downstream performance of models.<n>This paper investigates the extent to which this trend continues to hold for state-of-the-art reasoning models.
  arXiv  Detail & Related papers  (2025-07-21T07:49:32Z)
• Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questions [103.20281438405111]
  Multiple-choice question answering (MCQA) is a key competence of performant transformer language models.<n>We employ vocabulary projection and activation patching methods to localize key hidden states that encode relevant information for predicting the correct answer.<n>We show that subsequent layers increase the probability of the predicted answer symbol in vocabulary space, and that this probability increase is associated with a sparse set of attention heads with unique roles.
  arXiv  Detail & Related papers  (2024-07-21T00:10:23Z)
• "My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models [40.867655189493924]
  Open-ended nature of language generation makes evaluation of large language models (LLMs) challenging. One common evaluation approach uses multiple-choice questions (MCQ) to limit the response space. We evaluate how aligned first-token evaluation is with the text output along several dimensions.
  arXiv  Detail & Related papers  (2024-02-22T12:47:33Z)
• Toward Unsupervised Realistic Visual Question Answering [70.67698100148414]
  We study the problem of realistic VQA (RVQA), where a model has to reject unanswerable questions (UQs) and answer answerable ones (AQs) We first point out 2 drawbacks in current RVQA research, where (1) datasets contain too many unchallenging UQs and (2) a large number of annotated UQs are required for training. We propose a new testing dataset, RGQA, which combines AQs from an existing VQA dataset with around 29K human-annotated UQs. This combines pseudo UQs obtained by randomly pairing images and questions, with an
  arXiv  Detail & Related papers  (2023-03-09T06:58:29Z)
• Generating Correct Answers for Progressive Matrices Intelligence Tests [88.78821060331582]
  Raven's Progressive Matrices are multiple-choice intelligence tests, where one tries to complete the missing location in a $3times 3$ grid of abstract images. Previous attempts to address this test have focused solely on selecting the right answer out of the multiple choices. In this work, we focus, instead, on generating a correct answer given the grid, without seeing the choices, which is a harder task, by definition.
  arXiv  Detail & Related papers  (2020-11-01T13:21:07Z)
• Counterfactual Variable Control for Robust and Interpretable Question Answering [57.25261576239862]
  Deep neural network based question answering (QA) models are neither robust nor explainable in many cases. In this paper, we inspect such spurious "capability" of QA models using causal inference. We propose a novel approach called Counterfactual Variable Control (CVC) that explicitly mitigates any shortcut correlation.
  arXiv  Detail & Related papers  (2020-10-12T10:09:05Z)
• Crossing Variational Autoencoders for Answer Retrieval [50.17311961755684]
  Question-answer alignment and question/answer semantics are two important signals for learning the representations. We propose to cross variational auto-encoders by generating questions with aligned answers and generating answers with aligned questions.
  arXiv  Detail & Related papers  (2020-05-06T01:59:13Z)
• Robust Question Answering Through Sub-part Alignment [53.94003466761305]
  We model question answering as an alignment problem. We train our model on SQuAD v1.1 and test it on several adversarial and out-of-domain datasets.
  arXiv  Detail & Related papers  (2020-04-30T09:10:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.