LACE: Lattice Attention for Cross-thread Exploration

• URL: http://arxiv.org/abs/2604.15529v1
• Date: Thu, 16 Apr 2026 21:19:35 GMT
• Title: LACE: Lattice Attention for Cross-thread Exploration
• Abstract summary: We introduce LACE, a framework that transforms reasoning from a collection of independent trials into a coordinated, parallel process.<n>LACE allows concurrent reasoning paths to share intermediate insights and correct one another during inference.<n> Experiments show that this unified exploration substantially outperforms standard parallel search, improving reasoning accuracy by over 7 points.
• Score: 19.89703144721346
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Current large language models reason in isolation. Although it is common to sample multiple reasoning paths in parallel, these trajectories do not interact, and often fail in the same redundant ways. We introduce LACE, a framework that transforms reasoning from a collection of independent trials into a coordinated, parallel process. By repurposing the model architecture to enable cross-thread attention, LACE allows concurrent reasoning paths to share intermediate insights and correct one another during inference. A central challenge is the absence of natural training data that exhibits such collaborative behavior. We address this gap with a synthetic data pipeline that explicitly teaches models to communicate and error-correct across threads. Experiments show that this unified exploration substantially outperforms standard parallel search, improving reasoning accuracy by over 7 points. Our results suggest that large language models can be more effective when parallel reasoning paths are allowed to interact.

Related papers

• Parason: Revealing Subtask and Trial Parallelism in LLM Reasoning [20.249625251361238]
  We introduce Parason, which reveals and learns two forms of parallelism in large language models.<n>Parason converts sequential reasoning traces into structured parallel trajectories with a context-free grammar.<n>At inference time, Parason executes the learned parallel structure through tool calls, translating theoretical savings to real-world wall-clock acceleration.
  arXiv  Detail & Related papers  (2026-08-25T15:02:54Z)
• Semantic Alignment of AI Models: Concept Collapse, Checkpoint Dynamics, and Cross-Lingual Transfer [0.0]
  This work demonstrates how methods can be used to rigorously compare these spaces to low dimensional and interpretable baselines.<n>These multi-modal model alignment tests make it possible to track model adaptations test phrase understanding across multiple languages.
  arXiv  Detail & Related papers  (2026-08-03T01:41:04Z)
• Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching [66.39914384073145]
  We propose a self-consistency framework that turns cheap diffusion-sampled reasoning into a reusable pool of step-level candidates.<n>We find that step-level recombination is most beneficial on harder problems.<n>Our training-free framework improves average accuracy by up to 2 across six math and coding tasks.
  arXiv  Detail & Related papers  (2026-02-26T11:08:39Z)
• Consistency Is Not Always Correct: Towards Understanding the Role of Exploration in Post-Training Reasoning [75.79451512757844]
  Foundation models exhibit broad knowledge but limited task-specific reasoning.<n> RLVR and inference scaling motivate post-training strategies such as RLVR and inference scaling.<n>We show that RLVR induces a squeezing effect, reducing reasoning entropy and forgetting some correct paths.
  arXiv  Detail & Related papers  (2025-11-10T18:25:26Z)
• A Survey on Parallel Reasoning [58.66122129692264]
  We first present a formal definition of parallel reasoning and clarify its distinction from related concepts like Chain-of-Thought.<n>We then organize and discuss advanced techniques based on a novel taxonomy, including non-interactive reasoning, interactive reasoning, and efficiency-focused decoding strategies.<n>We highlight the core challenges of parallel reasoning and suggest potential directions for future research.
  arXiv  Detail & Related papers  (2025-10-14T05:42:19Z)
• Parallel-R1: Towards Parallel Thinking via Reinforcement Learning [65.68667585027232]
  Parallel thinking is a novel approach for enhancing the reasoning capabilities of large language models.<n>We propose textbfParallel-R1, the first reinforcement learning framework that enables parallel thinking behaviors.<n>Our framework employs a progressive curriculum that explicitly addresses the cold-start problem in training parallel thinking.
  arXiv  Detail & Related papers  (2025-09-09T17:59:35Z)
• Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity [23.225139930889522]
  This work critically examines whether intermediate token sequence length reflects or correlates with problem difficulty.<n>We train transformer models from scratch on derivational traces of the A* search algorithm.<n>We find that even for the simplest tasks, they often produce excessively long reasoning traces and sometimes fail to generate a solution.
  arXiv  Detail & Related papers  (2025-09-09T02:31:16Z)
• Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens [23.326813303795692]
  Chain-of-Thought (CoT) prompting has been shown to improve Large Language Model (LLM) performance on various tasks.<n>However, some initial findings suggest that CoT reasoning may be more superficial than it appears.
  arXiv  Detail & Related papers  (2025-08-02T04:37:28Z)
• Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models [49.84163262868945]
  Large language models have shown remarkable performance across a wide range of language tasks, owing to their exceptional capabilities in context modeling.<n>The most commonly used method of context modeling is full self-attention, as seen in standard decoder-only Transformers.<n>We propose parallel context encoding, which splits the context into sub-pieces and encodes them parallelly.
  arXiv  Detail & Related papers  (2024-12-21T09:04:51Z)
• Why think step by step? Reasoning emerges from the locality of experience [35.99870462573641]
  We investigate why and how chain-of-thought reasoning is useful in language models. We show that reasoning is effective when training data consists of overlapping local clusters of variables.
  arXiv  Detail & Related papers  (2023-04-07T21:04:03Z)
• Unification of HDP and LDA Models for Optimal Topic Clustering of Subject Specific Question Banks [55.41644538483948]
  An increase in the popularity of online courses would result in an increase in the number of course-related queries for academics. In order to reduce the time spent on answering each individual question, clustering them is an ideal choice. We use the Hierarchical Dirichlet Process to determine an optimal topic number input for our LDA model runs.
  arXiv  Detail & Related papers  (2020-10-04T18:21:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.