FrescoDiffusion: 4K Image-to-Video with Prior-Regularized Tiled Diffusion

• URL: http://arxiv.org/abs/2603.17555v1
• Date: Wed, 18 Mar 2026 10:02:37 GMT
• Title: FrescoDiffusion: 4K Image-to-Video with Prior-Regularized Tiled Diffusion
• Abstract summary: We introduce FrescoDiffusion, a training-free method for coherent large-format I2V generation from a single image.<n>For 4K generation, we compute per-tile noise predictions and fuse them with this reference at every diffusion timestep.<n>Experiments on the VBench-I2V dataset and our proposed fresco I2V dataset show improved global consistency and fidelity over tiled baselines.
• Score: 46.49480145234397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion-based image-to-video (I2V) models are increasingly effective, yet they struggle to scale to ultra-high-resolution inputs (e.g., 4K). Generating videos at the model's native resolution often loses fine-grained structure, whereas high-resolution tiled denoising preserves local detail but breaks global layout consistency. This failure mode is particularly severe in the fresco animation setting: monumental artworks containing many distinct characters, objects, and semantically different sub-scenes that must remain spatially coherent over time. We introduce FrescoDiffusion, a training-free method for coherent large-format I2V generation from a single complex image. The key idea is to augment tiled denoising with a precomputed latent prior: we first generate a low-resolution video at the underlying model resolution and upsample its latent trajectory to obtain a global reference that captures long-range temporal and spatial structure. For 4K generation, we compute per-tile noise predictions and fuse them with this reference at every diffusion timestep by minimizing a single weighted least-squares objective in model-output space. The objective combines a standard tile-merging criterion with our regularization term, yielding a closed-form fusion update that strengthens global coherence while retaining fine detail. We additionally provide a spatial regularization variable that enables region-level control over where motion is allowed. Experiments on the VBench-I2V dataset and our proposed fresco I2V dataset show improved global consistency and fidelity over tiled baselines, while being computationally efficient. Our regularization enables explicit controllability of the trade-off between creativity and consistency.

Related papers

• AtlasVid: Efficient Ultra-High-Resolution Long Video Generation via Decoupled Global-Local Modeling [44.836661735003474]
  We propose AtlaVid, a decoupled global-local framework for efficient ultra-high-resolution long video generation.<n>AtlaVid generates a low-resolution and low-FPS global semantic proxy via temporally scaled RoPE, thereby extending the temporal horizon without increasing the training token count.<n>Experiments show that AtlaVid substantially improves the efficiency of ultra-high-resolution long video generation, achieving high-quality UHR long video generation with 6x>0.9 speed and less training cost and even better performance than native 4K video generators.
  arXiv  Detail & Related papers  (2026-05-15T21:39:46Z)
• ScrollScape: Unlocking 32K Image Generation With Video Diffusion Priors [48.033666517340464]
  ScrollScape is a novel framework that reformulates EAR image synthesis into a continuous video generation process.<n>We show that ScrollScape significantly outperforms existing image-diffusion baselines by eliminating severe localized artifacts.
  arXiv  Detail & Related papers  (2026-03-25T13:03:02Z)
• RAYNOVA: Scale-Temporal Autoregressive World Modeling in Ray Space [51.441415833480505]
  RAYNOVA is a multiview world model for driving scenarios that employs a dual-causal autoregressive framework.<n>It constructs an isotropic-temporal representation across views, frames, and scales based on relative Plcker-ray positional encoding.
  arXiv  Detail & Related papers  (2026-02-24T08:41:40Z)
• Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers [95.68243351895107]
  We propose a holistic, video-centric paradigm named textbfLocal textbfDiffusion textbfForcing for textbfVideo textbfFrame textbfInterpolation (LDF-VFI)<n>Our framework is built upon an auto-regressive diffusion transformer that models the entire video sequence to ensure long-range temporal coherence.<n>LDF-VFI achieves state-of-the-art performance on challenging long-sequence benchmarks, demonstrating superior per
  arXiv  Detail & Related papers  (2026-01-21T12:58:52Z)
• Scale-DiT: Ultra-High-Resolution Image Generation with Hierarchical Local Attention [50.391914489898774]
  Scale-DiT is a new diffusion framework that introduces hierarchical local attention with low-resolution global guidance.<n>A lightweight LoRA adaptation bridges global and local pathways during denoising, ensuring consistency across structure and detail.<n>Experiments demonstrate that Scale-DiT achieves more than $2times$ faster inference and lower memory usage.
  arXiv  Detail & Related papers  (2025-10-18T03:15:26Z)
• STAR: Spatial-Temporal Augmentation with Text-to-Video Models for Real-World Video Super-Resolution [42.859188375578604]
  Image diffusion models have been adapted for real-world video superresolution to tackle over-smoothing issues in GAN-based methods.<n>These models struggle to maintain temporal consistency, as they are trained on static images.<n>We introduce a novel approach that leverages T2V models for real-world video super-resolution, achieving realistic spatial details and robust temporal consistency.
  arXiv  Detail & Related papers  (2025-01-06T12:36:21Z)
• High-Resolution Frame Interpolation with Patch-based Cascaded Diffusion [44.52838839928787]
  We introduce a patch-based cascaded pixel diffusion model for high resolution frame-to-frame processing.<n> HIFI excels at high-resolution images and complex repeated textures that require global context.<n>We introduce a new dataset, LaMoR, that focuses on particularly challenging cases.
  arXiv  Detail & Related papers  (2024-10-15T17:59:04Z)
• Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking [52.393359791978035]
  Motion2VecSets is a 4D diffusion model for dynamic surface reconstruction from point cloud sequences. We parameterize 4D dynamics with latent sets instead of using global latent codes. For more temporally-coherent object tracking, we synchronously denoise deformation latent sets and exchange information across multiple frames.
  arXiv  Detail & Related papers  (2024-01-12T15:05:08Z)
• Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis [59.10787643285506]
  Diffusion-based models have achieved state-of-the-art performance on text-to-image synthesis tasks. One critical limitation of these models is the low fidelity of generated images with respect to the text description. We propose a new text-to-image algorithm that adds explicit control over spatial-temporal cross-attention in diffusion models.
  arXiv  Detail & Related papers  (2023-04-07T23:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.