Reviving ConvNeXt for Efficient Convolutional Diffusion Models

• URL: http://arxiv.org/abs/2603.09408v1
• Date: Tue, 10 Mar 2026 09:24:30 GMT
• Title: Reviving ConvNeXt for Efficient Convolutional Diffusion Models
• Abstract summary: We introduce the fully convolutional diffusion model (FCDM), a model having a backbone similar to ConvNeXt.<n>We find that using only 50% of the FLOPs of DiT-XL/2, FCDM-XL achieves competitive performance with 7$times$ and 7.5$times$ fewer training steps at 256$times$256 and 512$times$512 resolutions.
• Score: 39.295834664366005
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Recent diffusion models increasingly favor Transformer backbones, motivated by the remarkable scalability of fully attentional architectures. Yet the locality bias, parameter efficiency, and hardware friendliness--the attributes that established ConvNets as the efficient vision backbone--have seen limited exploration in modern generative modeling. Here we introduce the fully convolutional diffusion model (FCDM), a model having a backbone similar to ConvNeXt, but designed for conditional diffusion modeling. We find that using only 50% of the FLOPs of DiT-XL/2, FCDM-XL achieves competitive performance with 7$\times$ and 7.5$\times$ fewer training steps at 256$\times$256 and 512$\times$512 resolutions, respectively. Remarkably, FCDM-XL can be trained on a 4-GPU system, highlighting the exceptional training efficiency of our architecture. Our results demonstrate that modern convolutional designs provide a competitive and highly efficient alternative for scaling diffusion models, reviving ConvNeXt as a simple yet powerful building block for efficient generative modeling.

Related papers

• Recursive Block-Diagonal Coupling for Resource-Efficient Training of Vision Models [16.54656665398701]
  Training high-capacity vision models from scratch requires substantial computational resources.<n>We propose an efficient training protocol, RBDC, that builds wide models by coupling in a parameter-free block-diagonal way.<n>Our protocol shows a much better efficiency than models trained from scratch with the standard protocol, yielding 30% FLOPs reduction at similar test accuracies.
  arXiv  Detail & Related papers  (2026-05-22T14:08:22Z)
• On the Design of One-step Diffusion via Shortcutting Flow Paths [78.72016001375935]
  We propose a common design framework for representative shortcut models.<n>With our proposed improvements, the resulting one-step model achieves a new state-of-the-art FID50k of 2.85 on ImageNet-256x256.<n>Remarkably, the model requires no pre-training, distillation, or curriculum learning.
  arXiv  Detail & Related papers  (2025-12-03T09:28:29Z)
• E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources [12.244453688491731]
  Efficient Multimodal Diffusion Transformer (E-MMDiT) is an efficient and lightweight multimodal diffusion model with only 304M parameters for fast image synthesis.<n>Our model for 512px generation, trained with only 25M public data in 1.5 days on a single node of 8 AMD MI300X GPUs, achieves 0.66 on GenEval and easily reaches to 0.72 with some post-training techniques such as GRPO.
  arXiv  Detail & Related papers  (2025-10-31T03:13:08Z)
• Home-made Diffusion Model from Scratch to Hatch [0.9383683724544296]
  Home-made Diffusion Model (HDM) is an efficient yet powerful text-to-image diffusion model optimized for training on consumer-grade hardware.<n>HDM achieves competitive 1024x1024 generation quality while maintaining a remarkably low training cost of $535-620.
  arXiv  Detail & Related papers  (2025-09-07T14:21:57Z)
• DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modeling [41.128461167768805]
  Diffusion Transformer (DiT) is a promising diffusion model for visual generation but incurs significant computational overhead.<n>We revisit convolution as an alternative building block for constructing efficient and expressive diffusion models.<n>DiCo-XL achieves an FID of 2.05 at 256x256 resolution and 2.53 at 512x512, with a 2.7x and 3.1x speedup over DiT-XL/2, respectively.
  arXiv  Detail & Related papers  (2025-05-16T12:54:04Z)
• Masked Autoencoders Are Effective Tokenizers for Diffusion Models [56.08109308294133]
  MAETok is an autoencoder that learns semantically rich latent space while maintaining reconstruction fidelity.<n>MaETok achieves significant practical improvements, enabling a gFID of 1.69 with 76x faster training and 31x higher inference throughput for 512x512 generation.
  arXiv  Detail & Related papers  (2025-02-05T18:42:04Z)
• One-Step Diffusion Distillation through Score Implicit Matching [74.91234358410281]
  We present Score Implicit Matching (SIM) a new approach to distilling pre-trained diffusion models into single-step generator models. SIM shows strong empirical performances for one-step generators. By applying SIM to a leading transformer-based diffusion model, we distill a single-step generator for text-to-image generation.
  arXiv  Detail & Related papers  (2024-10-22T08:17:20Z)
• SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow [24.213303324584906]
  We develop small, efficient one-step diffusion models based on the powerful rectified flow framework. We train a one-step diffusion model with an FID of 5.02 and 15.7M parameters, outperforming the previous state-of-the-art one-step diffusion model.
  arXiv  Detail & Related papers  (2024-07-17T16:38:45Z)
• TerDiT: Ternary Diffusion Models with Transformers [88.03738506648291]
  TerDiT is the first quantization-aware training scheme for low-bit diffusion transformer models.<n>We focus on the ternarization of DiT networks, with model sizes ranging from 600M to 4.2B, and image resolution from 256$times$256 to 512$times$512.
  arXiv  Detail & Related papers  (2024-05-23T17:57:24Z)
• Consistency Models [89.68380014789861]
  We propose a new family of models that generate high quality samples by directly mapping noise to data. They support fast one-step generation by design, while still allowing multistep sampling to trade compute for sample quality. They also support zero-shot data editing, such as image inpainting, colorization, and super-resolution, without requiring explicit training.
  arXiv  Detail & Related papers  (2023-03-02T18:30:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.