ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generation

• URL: http://arxiv.org/abs/2508.17345v1
• Date: Sun, 24 Aug 2025 13:03:02 GMT
• Title: ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generation
• Abstract summary: Shortlisting Model (SLM) is a novel simplex-based diffusion model inspired by progressive candidate pruning.<n>SLM operates on simplex centroids, reducing generation complexity and enhancing scalability.
• Score: 47.20130201282587
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generative modeling of discrete variables is challenging yet crucial for applications in natural language processing and biological sequence design. We introduce the Shortlisting Model (SLM), a novel simplex-based diffusion model inspired by progressive candidate pruning. SLM operates on simplex centroids, reducing generation complexity and enhancing scalability. Additionally, SLM incorporates a flexible implementation of classifier-free guidance, enhancing unconditional generation performance. Extensive experiments on DNA promoter and enhancer design, protein design, character-level and large-vocabulary language modeling demonstrate the competitive performance and strong potential of SLM. Our code can be found at https://github.com/GenSI-THUAIR/SLM

Related papers

• Diffutron: A Masked Diffusion Language Model for Turkish Language [0.0]
  We introduce $textitDiffutron$, a masked diffusion language model specifically designed for Turkish.<n>We employ a progressive instruction-tuning strategy, sequentially adapting the model on general and task-specific instruction sets.<n>These findings validate the effectiveness of masked diffusion modeling combined with multi-stage tuning for non-autoregressive text generation in Turkish.
  arXiv  Detail & Related papers  (2026-03-20T19:52:37Z)
• Insertion Based Sequence Generation with Learnable Order Dynamics [31.889457692631023]
  We show that joint training of the target order dynamics and the generator is tractable without the need for numerical simulation.<n>As the generative insertion model, we use a variable length masked diffusion model, which generates by inserting and filling mask tokens.<n>We find that the learned order dynamics leads to an increase in the number of valid molecules generated and improved quality, when compared to uniform order dynamics.
  arXiv  Detail & Related papers  (2026-02-21T02:53:55Z)
• ThinkGen: Generalized Thinking for Visual Generation [97.19923474851987]
  ThinkGen is a think-driven visual generation framework that explicitly leverages Chain-of-Thought (CoT) reasoning in various generation scenarios.<n>We propose a separable GRPO-based training paradigm, alternating reinforcement learning between the MLLM and DiT modules.<n>Experiments demonstrate that ThinkGen achieves robust, state-of-the-art performance across multiple generation benchmarks.
  arXiv  Detail & Related papers  (2025-12-29T16:08:50Z)
• Unveiling the Potential of Diffusion Large Language Model in Controllable Generation [36.05635830551406]
  Even state-of-the-art autoregressive Large Language Models (LLMs) exhibit unreliability when required to generate structured output.<n>Inspired by the current new diffusion-based large language models (dLLM), we realize that the architectural difference may be the key to unlock next-level controllable generation.<n>We propose Self-adaptive Scaffolding, a novel framework that enables dLLM to stably generate reliable structured outputs.
  arXiv  Detail & Related papers  (2025-07-06T18:41:34Z)
• Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows [46.673228292287895]
  We propose a novel framework that employs transformer-based autoregressive normalizing flows to model continuous representations.<n>This approach unlocks substantial flexibility, enabling the construction of models that can capture global bi-directional context.<n>We propose new mixture-based coupling transformations designed to capture complex dependencies within the latent space shaped by discrete data.
  arXiv  Detail & Related papers  (2025-07-01T04:51:25Z)
• CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design [11.639825726501659]
  Large language models (LLMs) can autonomously discover high-performings at a fraction of the traditional cost.<n>We propose a hybrid framework that combines verbal and numerical guidance.<n>Our method outperforms state-of-the-art (SOTA) baselines across various optimization tasks.
  arXiv  Detail & Related papers  (2025-05-18T07:48:47Z)
• SynerGen-VL: Towards Synergistic Image Understanding and Generation with Vision Experts and Token Folding [66.74446220401296]
  We propose SynerGen-VL, a simple yet powerful encoder-free MLLM capable of both image understanding and generation.<n>We introduce the token folding mechanism and the vision-expert-based progressive alignment pretraining strategy, which effectively support high-resolution image understanding.<n>Our code and models shall be released.
  arXiv  Detail & Related papers  (2024-12-12T18:59:26Z)
• Multimodal Latent Language Modeling with Next-Token Diffusion [111.93906046452125]
  Multimodal generative models require a unified approach to handle both discrete data (e.g., text and code) and continuous data (e.g., image, audio, video)<n>We propose Latent Language Modeling (LatentLM), which seamlessly integrates continuous and discrete data using causal Transformers.
  arXiv  Detail & Related papers  (2024-12-11T18:57:32Z)
• Genetic Instruct: Scaling up Synthetic Generation of Coding Instructions for Large Language Models [59.60208063956459]
  Large Language Models (LLMs) require high quality instruction data for effective alignment.<n>We present Genetic-Instruct, a scalable algorithm for synthesizing large-scale, high quality coding instructions.
  arXiv  Detail & Related papers  (2024-07-29T20:42:59Z)
• Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning [115.79349923044663]
  Few-shot class-incremental learning (FSCIL) aims to incrementally learn novel classes from limited examples.<n>Existing methods face a critical dilemma: static architectures rely on a fixed parameter space to learn from data that arrive sequentially, prone to overfitting to the current session.<n>In this study, we explore the potential of Selective State Space Models (SSMs) for FSCIL.
  arXiv  Detail & Related papers  (2024-07-08T17:09:39Z)
• FiLM: Fill-in Language Models for Any-Order Generation [71.42044325886194]
  Fill-in Language Model (FiLM) is a new language modeling approach that allows for flexible generation at any position without adhering to a specific generation order. During inference, FiLM can seamlessly insert missing phrases, sentences, or paragraphs. FiLM outperforms existing infilling methods that rely on left-to-right language models trained on rearranged text segments.
  arXiv  Detail & Related papers  (2023-10-15T19:37:39Z)
• EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs [26.462946557604177]
  EasyGen is designed to enhance multimodal understanding and generation by harnessing the capabilities of diffusion models and large language models (LLMs) Easygen achieves text generation by training a projection layer linking BiDiffuser and an LLM, and facilities image generation by training an adapter to align the LLM's text space with the BiDiffuser's image space.
  arXiv  Detail & Related papers  (2023-10-13T08:38:56Z)
• Sparse Modular Activation for Efficient Sequence Modeling [94.11125833685583]
  Recent models combining Linear State Space Models with self-attention mechanisms have demonstrated impressive results across a range of sequence modeling tasks. Current approaches apply attention modules statically and uniformly to all elements in the input sequences, leading to sub-optimal quality-efficiency trade-offs. We introduce Sparse Modular Activation (SMA), a general mechanism enabling neural networks to sparsely activate sub-modules for sequence elements in a differentiable manner.
  arXiv  Detail & Related papers  (2023-06-19T23:10:02Z)
• Diffusion-LM Improves Controllable Text Generation [80.50044830018442]
  Controlling the behavior of language models (LMs) without re-training is a major open problem in natural language generation. We develop a new non-autoregressive language model based on continuous diffusions that we call Diffusion-LM. We demonstrate successful control of Diffusion-LM for six challenging fine-grained control tasks, significantly outperforming prior work.
  arXiv  Detail & Related papers  (2022-05-27T20:12:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.