Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning

• URL: http://arxiv.org/abs/2603.09184v1
• Date: Tue, 10 Mar 2026 04:45:26 GMT
• Title: Latent-DARM: Bridging Discrete Diffusion And Autoregressive Models For Reasoning
• Abstract summary: We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors)<n>Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average.<n>Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget.
• Score: 40.43847739412248
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Most multi-agent systems rely exclusively on autoregressive language models (ARMs) that are based on sequential generation. Although effective for fluent text, ARMs limit global reasoning and plan revision. On the other hand, Discrete Diffusion Language Models (DDLMs) enable non-sequential, globally revisable generation and have shown strong planning capabilities, but their limited text fluency hinders direct collaboration with ARMs. We introduce Latent-DARM, a latent-space communication framework bridging DDLM (planners) and ARM (executors), maximizing collaborative benefits. Across mathematical, scientific, and commonsense reasoning benchmarks, Latent-DARM outperforms text-based interfaces on average, improving accuracy from 27.0% to 36.0% on DART-5 and from 0.0% to 14.0% on AIME2024. Latent-DARM approaches the results of state-of-the-art reasoning models while using less than 2.2% of its token budget. This work advances multi-agent collaboration among agents with heterogeneous models.

Related papers

• ARM: Role-Conditioned Neuron Transplantation for Training-Free Generalist LLM Agent Merging [51.409102048965394]
  Agent-Role Merging (ARM) is an activation-guided, role-conditioned neuron transplantation method for model merging in LLM agents.<n>ARM improves existing merging methods from static natural language tasks to multi-turn agent scenarios.
  arXiv  Detail & Related papers  (2026-01-12T08:31:53Z)
• Latent Collaboration in Multi-Agent Systems [140.51506923969345]
  Multi-agent systems (MAS) extend from independent single-model reasoning to coordinative system-level intelligence.<n>We introduce LatentMAS, an end-to-end training-free framework that enables pure latent collaboration among LLM agents.
  arXiv  Detail & Related papers  (2025-11-25T18:56:57Z)
• Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoning [40.69037032768994]
  Current autoregressive language models (ARMs) achieve high accuracy but require long token sequences, making them costly.<n>DDLMs enable parallel and flexible generation within a fixed number of steps.<n>We present a study exploring hybrid architectures that couple DDLMs with ARMs to assess whether their collaboration can yield complementary benefits.
  arXiv  Detail & Related papers  (2025-10-17T02:16:19Z)
• Beyond Next-Token Prediction: A Performance Characterization of Diffusion versus Autoregressive Language Models [82.87985794856803]
  Large Language Models (LLMs) have achieved state-of-the-art performance on a broad range of Natural Language Processing (NLP) tasks.<n>Recently, Diffusion Language Models (DLMs) have emerged as a promising alternative architecture.
  arXiv  Detail & Related papers  (2025-10-05T10:50:52Z)
• Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding [55.2480439325792]
  In arbitrary-order language models, it is an open question how to sample tokens in parallel from the correct joint distribution.<n>We find that a different class of models, any-subset autoregressive models (AS-ARMs), holds the solution.<n>We show that AS-ARMs achieve state-of-the-art performance among sub-200M parameter models on infilling benchmark tasks, and nearly match the performance of models 50X larger on code generation.
  arXiv  Detail & Related papers  (2025-04-29T06:33:13Z)
• Large Language Diffusion Models [93.26422905620008]
  Large language models (LLMs) are widely regarded as relying on autoregressive models (ARMs)<n>We introduce LLaDA, a diffusion model trained from scratch under the pre-training and supervised fine-tuning paradigm.<n>Across extensive benchmarks on general tasks, math, code, and so on, LLaDA demonstrates strong scalability and performs comparably to our self-constructed ARM baselines.
  arXiv  Detail & Related papers  (2025-02-14T08:23:51Z)
• Scaling up Masked Diffusion Models on Text [43.16800764711572]
  Masked diffusion models (MDMs) have shown promise in language modeling.<n>This paper establishes the first scaling law for MDMs.<n>We train a family of MDMs with up to 1.1 billion (B) parameters to evaluate their performance against larger sizes.
  arXiv  Detail & Related papers  (2024-10-24T08:01:22Z)
• ACDC: Autoregressive Coherent Multimodal Generation using Diffusion Correction [55.03585818289934]
  Autoregressive models (ARMs) and diffusion models (DMs) represent two leading paradigms in generative modeling. We introduce Autoregressive Coherent multimodal generation with Diffusion Correction (ACDC) ACDC combines the strengths of both ARMs and DMs at the inference stage without the need for additional fine-tuning.
  arXiv  Detail & Related papers  (2024-10-07T03:22:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.