Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning

• URL: http://arxiv.org/abs/2607.21653v1
• Date: Wed, 22 Jul 2026 18:06:15 GMT
• Title: Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning
• Abstract summary: Molt is a PyTorch-native training framework built to keep that cost small.<n>One asynchronous loop trains multimodal and mixture-of-experts policies.<n>Molt is statistically comparable to a state-of-the-art Megatron-based stack.
• Score: 56.80047499513455
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Agentic reinforcement learning research is constant algorithm modification, new estimators, new pipeline stages, new rollout schemes, and in mainstream frameworks each change threads through layers of trainer, distributed backend, and rollout glue: the cost lands on the researcher at every iteration. Molt is a PyTorch-native training framework built to keep that cost small: a codebase compact and clean enough for a researcher to hold in their head, and for an AI coding assistant to read and reason about in its entirety, so the algorithm flow can be traced and changed end to end. The agent is an ordinary program, and one asynchronous loop trains multimodal and mixture-of-experts policies while never training on a token it did not generate, consistent in tokens, policy versions, and model semantics. Leanness does not cost performance: under a matched, fully asynchronous protocol, Molt is statistically comparable to a state-of-the-art Megatron-based stack. Molt is open source and provides recipes and containers at https://github.com/NVIDIA-NeMo/labs-molt.

Related papers

• AgentFold: Closed-Loop Agentic Search for Protein Folding Model Design [27.154390712714264]
  We present AgentFold, a framework that formulates folding-model development as a closed-loop search over executable code variants.<n>Under a matched computational budget, AgentFold improves the best lDDT by 7.5% over independent Codex proposals.<n>The resulting intervention traces reveal recurring empirical design patterns.
  arXiv  Detail & Related papers  (2026-08-27T07:38:16Z)
• Dynamic Mixture of Latent Memories for Self-Evolving Agents [57.20419419302731]
  MoLEM is a generative mixture of latent memory framework based on a dynamic mixture-of-experts (MoE)<n>We train the framework on continual-learning sequences spanning math, science, and code domains.<n>After the full continual-learning sequence, our method improves the average accuracy by 10.40% over the Vanilla pretrained baseline.
  arXiv  Detail & Related papers  (2026-05-21T03:35:10Z)
• ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation [55.947962672433675]
  ChipMATE is the first self-trained multi-agent framework for RTL generation.<n>Inspired by industrial practice, ChipMATE pairs a Verilog agent with a Python reference-model agent that mutually verify each other's outputs.<n>ChipMATE achieves 75.0% and 80.1% pass@1 on VerilogEval V2 with 4B and 9B base models.
  arXiv  Detail & Related papers  (2026-05-13T01:04:21Z)
• Model Merging: Foundations and Algorithms [4.528573838858818]
  This thesis studies model merging, combining independently trained neural networks directly in weight space.<n>In the single-task setting, we introduce C$2$M$3$, a cycle-consistent merging algorithm based on Frank-Wolfe optimization.<n>In the multi-task setting, we first develop a theoretical account of task vectors as approximate gradients.<n>We then present MASS, an input-adaptive routing method that uses TSV geometry to select task-relevant subspaces at inference time.
  arXiv  Detail & Related papers  (2026-05-02T19:06:35Z)
• MIP Candy: A Modular PyTorch Framework for Medical Image Processing [4.024630879799288]
  MIPCandy is a PyTorch-based framework designed specifically for medical image processing.<n>Central to the design is $textttLayerT$, a deferred configuration mechanism that enables runtime substitution of convolution, normalization, and activation modules.<n>MIPCandy is open-source under the Apache-2.0 license and requires Python3.12 or later.
  arXiv  Detail & Related papers  (2026-02-24T15:55:04Z)
• MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models [28.79185891706149]
  Diffusion language models suffer from a key discrepancy between training and inference.<n>We propose a novel Masked Diffusion Policy Optimization (MDPO) to exploit the Markov property diffusion.<n>Our findings establish great potential for investigating the discrepancy between pre-training and inference of MDLMs.
  arXiv  Detail & Related papers  (2025-08-18T17:58:13Z)
• MALT: Improving Reasoning with Multi-Agent LLM Training [67.76186488361685]
  MALT (Multi-Agent LLM Training) is a novel post-training strategy that divides the reasoning process into generation, verification, and refinement steps.<n>On MATH, GSM8K, and CSQA, MALT surpasses the same baseline LLM with a relative improvement of 15.66%, 7.42%, and 9.40% respectively.
  arXiv  Detail & Related papers  (2024-12-02T19:30:36Z)
• Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control [66.78146440275093]
  Learned retrieval (LSR) is a family of neural methods that encode queries and documents into sparse lexical vectors. We explore the application of LSR to the multi-modal domain, with a focus on text-image retrieval. Current approaches like LexLIP and STAIR require complex multi-step training on massive datasets. Our proposed approach efficiently transforms dense vectors from a frozen dense model into sparse lexical vectors.
  arXiv  Detail & Related papers  (2024-02-27T14:21:56Z)
• MADiff: Offline Multi-agent Learning with Diffusion Models [79.18130544233794]
  MADiff is a diffusion-based multi-agent learning framework.<n>It works as both a decentralized policy and a centralized controller.<n>Our experiments demonstrate that MADiff outperforms baseline algorithms across various multi-agent learning tasks.
  arXiv  Detail & Related papers  (2023-05-27T02:14:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.