Demystifying Numerosity in Diffusion Models -- Limitations and Remedies

• URL: http://arxiv.org/abs/2510.11117v1
• Date: Mon, 13 Oct 2025 08:07:24 GMT
• Title: Demystifying Numerosity in Diffusion Models -- Limitations and Remedies
• Abstract summary: Numerosity remains a challenge for state-of-the-art text-to-image generation models like FLUX and GPT-4o.<n>Can diffusion models inherently generate the correct number of objects specified by a textual prompt simply by scaling up the dataset and model size?<n>We construct a clean synthetic numerosity benchmark comprising two complementary datasets: GrayCount250 for controlled scaling studies, and NaturalCount6 featuring complex naturalistic scenes.
• Score: 29.067276405073283
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Numerosity remains a challenge for state-of-the-art text-to-image generation models like FLUX and GPT-4o, which often fail to accurately follow counting instructions in text prompts. In this paper, we aim to study a fundamental yet often overlooked question: Can diffusion models inherently generate the correct number of objects specified by a textual prompt simply by scaling up the dataset and model size? To enable rigorous and reproducible evaluation, we construct a clean synthetic numerosity benchmark comprising two complementary datasets: GrayCount250 for controlled scaling studies, and NaturalCount6 featuring complex naturalistic scenes. Second, we empirically show that the scaling hypothesis does not hold: larger models and datasets alone fail to improve counting accuracy on our benchmark. Our analysis identifies a key reason: diffusion models tend to rely heavily on the noise initialization rather than the explicit numerosity specified in the prompt. We observe that noise priors exhibit biases toward specific object counts. In addition, we propose an effective strategy for controlling numerosity by injecting count-aware layout information into the noise prior. Our method achieves significant gains, improving accuracy on GrayCount250 from 20.0\% to 85.3\% and on NaturalCount6 from 74.8\% to 86.3\%, demonstrating effective generalization across settings.

Related papers

• Learning More from Less: Unlocking Internal Representations for Benchmark Compression [37.69575776639016]
  We introduce REPCORE, which aligns heterogeneous hidden states into a unified latent space to construct representative coresets.<n>Experiments on five benchmarks and over 200 models show consistent gains over output-based baselines in ranking correlation and estimation accuracy.
  arXiv  Detail & Related papers  (2026-01-31T13:11:39Z)
• CountSteer: Steering Attention for Object Counting in Diffusion Models [21.46393378362606]
  Text-to-image diffusion models generate realistic and coherent images but often fail to follow numerical instructions in text.<n>We introduce CountSteer, a training-free method that improves generation of specified object counts by steering the model's cross-attention hidden states during inference.
  arXiv  Detail & Related papers  (2025-11-14T12:52:11Z)
• Accelerated Test-Time Scaling with Model-Free Speculative Sampling [58.69141724095398]
  We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach.<n>We show that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding.<n>As a model-free approach, STAND can be applied to any existing language model without additional training.
  arXiv  Detail & Related papers  (2025-06-05T07:31:18Z)
• CountDiffusion: Text-to-Image Synthesis with Training-Free Counting-Guidance Diffusion [82.82885671486795]
  We propose CountDiffusion, a training-free framework aiming at generating images with correct object quantity from textual descriptions.<n>The proposed CountDiffusion can be plugged into any diffusion-based text-to-image (T2I) generation models without further training.
  arXiv  Detail & Related papers  (2025-05-07T11:47:35Z)
• Simple and Critical Iterative Denoising: A Recasting of Discrete Diffusion in Graph Generation [0.0]
  dependencies between intermediate noisy states lead to error accumulation and propagation during the reverse denoising process.<n>We propose a novel framework called Simple Iterative Denoising, which simplifies discrete diffusion and circumvents the issue.<n>Our empirical evaluations demonstrate that the proposed method significantly outperforms existing discrete diffusion baselines in graph generation tasks.
  arXiv  Detail & Related papers  (2025-03-27T15:08:58Z)
• Text-to-Image Diffusion Models Cannot Count, and Prompt Refinement Cannot Help [18.70937620674227]
  We introduce T2ICountBench, a novel benchmark designed to rigorously evaluate the counting ability of state-of-the-art text-to-image diffusion models.<n>Our evaluations reveal that all state-of-the-art diffusion models fail to generate the correct number of objects, with accuracy dropping significantly as the number of objects increases.
  arXiv  Detail & Related papers  (2025-03-10T03:28:18Z)
• Unique Rashomon Sets for Robust Active Learning [0.0]
  We introduce UNique Rashomon Ensembled Active Learning (UNREAL)<n>UNREAL selectively ensembles models from the Rashomon set, which is the set of nearly optimal models.<n>We show that UNREAL achieves faster theoretical convergence rates than traditional active learning approaches.
  arXiv  Detail & Related papers  (2025-03-09T20:50:34Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Spatial Reasoning with Denoising Models [49.83744014336816]
  We introduce a framework to perform reasoning over sets of continuous variables via denoising generative models.<n>For the first time, that order of generation can successfully be predicted by the denoising network itself.<n>Using these findings, we can increase the accuracy of specific reasoning tasks from 1% to >50%.
  arXiv  Detail & Related papers  (2025-02-28T14:08:30Z)
• Dimension-free Score Matching and Time Bootstrapping for Diffusion Models [11.743167854433306]
  Diffusion models generate samples by estimating the score function of the target distribution at various noise levels.<n>In this work, we establish the first (nearly) dimension-free sample bounds complexity for learning these score functions.<n>A key aspect of our analysis is the use of a single function approximator to jointly estimate scores across noise levels.
  arXiv  Detail & Related papers  (2025-02-14T18:32:22Z)
• Towards Pattern-aware Data Augmentation for Temporal Knowledge Graph Completion [18.51546761241817]
  We introduce Booster, the first data augmentation strategy for temporal knowledge graphs.<n>We propose a hierarchical scoring algorithm based on triadic closures within TKGs.<n>We also propose a two-stage training approach to identify samples that deviate from the model's preferred patterns.
  arXiv  Detail & Related papers  (2024-12-31T03:47:19Z)
• Fine-tuning Language Models for Factuality [96.5203774943198]
  Large pre-trained language models (LLMs) have led to their widespread use, sometimes even as a replacement for traditional search engines. Yet language models are prone to making convincing but factually inaccurate claims, often referred to as 'hallucinations' In this work, we fine-tune language models to be more factual, without human labeling.
  arXiv  Detail & Related papers  (2023-11-14T18:59:15Z)
• Towards Better Certified Segmentation via Diffusion Models [62.21617614504225]
  segmentation models can be vulnerable to adversarial perturbations, which hinders their use in critical-decision systems like healthcare or autonomous driving. Recently, randomized smoothing has been proposed to certify segmentation predictions by adding Gaussian noise to the input to obtain theoretical guarantees. In this paper, we address the problem of certifying segmentation prediction using a combination of randomized smoothing and diffusion models.
  arXiv  Detail & Related papers  (2023-06-16T16:30:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.