GriDiT: Factorized Grid-Based Diffusion for Efficient Long Image Sequence Generation

• URL: http://arxiv.org/abs/2512.21276v1
• Date: Wed, 24 Dec 2025 16:46:04 GMT
• Title: GriDiT: Factorized Grid-Based Diffusion for Efficient Long Image Sequence Generation
• Authors: Snehal Singh Tomar, Alexandros Graikos, Arjun Krishna, Dimitris Samaras, Klaus Mueller,
• Abstract summary: We train a generative model solely on grid images comprising subsampled frames.<n>We learn to generate image sequences, using the strong self-attention mechanism of the Diffusion Transformer (DiT) to capture correlations between frames.<n>Our method consistently outperforms SoTA in quality and inference speed (at least twice-as-fast) across datasets.
• Score: 77.13582457917418
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Modern deep learning methods typically treat image sequences as large tensors of sequentially stacked frames. However, is this straightforward representation ideal given the current state-of-the-art (SoTA)? In this work, we address this question in the context of generative models and aim to devise a more effective way of modeling image sequence data. Observing the inefficiencies and bottlenecks of current SoTA image sequence generation methods, we showcase that rather than working with large tensors, we can improve the generation process by factorizing it into first generating the coarse sequence at low resolution and then refining the individual frames at high resolution. We train a generative model solely on grid images comprising subsampled frames. Yet, we learn to generate image sequences, using the strong self-attention mechanism of the Diffusion Transformer (DiT) to capture correlations between frames. In effect, our formulation extends a 2D image generator to operate as a low-resolution 3D image-sequence generator without introducing any architectural modifications. Subsequently, we super-resolve each frame individually to add the sequence-independent high-resolution details. This approach offers several advantages and can overcome key limitations of the SoTA in this domain. Compared to existing image sequence generation models, our method achieves superior synthesis quality and improved coherence across sequences. It also delivers high-fidelity generation of arbitrary-length sequences and increased efficiency in inference time and training data usage. Furthermore, our straightforward formulation enables our method to generalize effectively across diverse data domains, which typically require additional priors and supervision to model in a generative context. Our method consistently outperforms SoTA in quality and inference speed (at least twice-as-fast) across datasets.

Related papers

• UniRef-Image-Edit: Towards Scalable and Consistent Multi-Reference Image Editing [33.64590153603506]
  We present UniRef-Image-Edit, a high-performance multi-modal generation system.<n>It unifies single-image editing and multi-image composition within a single framework.
  arXiv  Detail & Related papers  (2026-02-15T15:24:03Z)
• Next Visual Granularity Generation [58.22272282205028]
  We propose a novel approach to image generation by decomposing an image into a structured sequence.<n>Next Visual Granularity (NVG) generation framework generates a visual granularity sequence.<n>We train a series of NVG models for class-conditional image generation on the ImageNet dataset and observe clear scaling behavior.
  arXiv  Detail & Related papers  (2025-08-18T10:47:37Z)
• Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series [7.201938834736084]
  We propose a unified generative model for varying-length time series. We employ invertible transforms such as the delay embedding and the short-time Fourier transform. We show that our approach achieves consistently state-of-the-art results against strong baselines.
  arXiv  Detail & Related papers  (2024-10-25T13:06:18Z)
• Fast constrained sampling in pre-trained diffusion models [80.99262780028015]
  We propose an algorithm that enables fast, high-quality generation under arbitrary constraints.<n>Our approach produces results that rival or surpass the state-of-the-art training-free inference methods.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer Normalization [26.926712014346432]
  This paper presents innovative enhancements to diffusion models by integrating a novel multi-resolution network and time-dependent layer normalization.<n>Our method's efficacy is demonstrated on the class-conditional ImageNet generation benchmark, setting new state-of-the-art FID scores of 1.70 on ImageNet 256 x 256 and 2.89 on ImageNet 512 x 512.
  arXiv  Detail & Related papers  (2024-06-13T17:59:58Z)
• Active Generation for Image Classification [45.93535669217115]
  We propose to address the efficiency of image generation by focusing on the specific needs and characteristics of the model. With a central tenet of active learning, our method, named ActGen, takes a training-aware approach to image generation.
  arXiv  Detail & Related papers  (2024-03-11T08:45:31Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• Progressive Text-to-Image Generation [40.09326229583334]
  We present a progressive model for high-fidelity text-to-image generation. The proposed method takes effect by creating new image tokens from coarse to fine based on the existing context. The resulting coarse-to-fine hierarchy makes the image generation process intuitive and interpretable.
  arXiv  Detail & Related papers  (2022-10-05T14:27:20Z)
• DynaST: Dynamic Sparse Transformer for Exemplar-Guided Image Generation [56.514462874501675]
  We propose a dynamic sparse attention based Transformer model to achieve fine-level matching with favorable efficiency. The heart of our approach is a novel dynamic-attention unit, dedicated to covering the variation on the optimal number of tokens one position should focus on. Experiments on three applications, pose-guided person image generation, edge-based face synthesis, and undistorted image style transfer, demonstrate that DynaST achieves superior performance in local details.
  arXiv  Detail & Related papers  (2022-07-13T11:12:03Z)
• NP-DRAW: A Non-Parametric Structured Latent Variable Modelfor Image Generation [139.8037697822064]
  We present a non-parametric structured latent variable model for image generation, called NP-DRAW. It sequentially draws on a latent canvas in a part-by-part fashion and then decodes the image from the canvas.
  arXiv  Detail & Related papers  (2021-06-25T05:17:55Z)
• Bayesian Image Reconstruction using Deep Generative Models [7.012708932320081]
  In this work, we leverage state-of-the-art (SOTA) generative models for building powerful image priors. Our method, called Bayesian Reconstruction through Generative Models (BRGM), uses a single pre-trained generator model to solve different image restoration tasks.
  arXiv  Detail & Related papers  (2020-12-08T17:11:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.