DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editing
- URL: http://arxiv.org/abs/2603.28713v1
- Date: Mon, 30 Mar 2026 17:30:25 GMT
- Title: DreamLite: A Lightweight On-Device Unified Model for Image Generation and Editing
- Abstract summary: We propose a compact unified on-device diffusion model (0.39B) that supports both T2I generation and text-guided image editing within a single network.<n>DreamLite is built on a pruned mobile U-Net backbone and unifies images as input, using a (target | blank) configuration for generation tasks and (target | source) for editing tasks.<n>After high-quality SFT and reinforcement learning, DreamLite achieves GenEval (0.72) for image generation and ImgEdit (4.11) for image editing, outperforming existing on-device models.
- Score: 12.515161196847442
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Diffusion models have made significant progress in both text-to-image (T2I) generation and text-guided image editing. However, these models are typically built with billions of parameters, leading to high latency and increased deployment challenges. While on-device diffusion models improve efficiency, they largely focus on T2I generation and lack support for image editing. In this paper, we propose DreamLite, a compact unified on-device diffusion model (0.39B) that supports both T2I generation and text-guided image editing within a single network. DreamLite is built on a pruned mobile U-Net backbone and unifies conditioning through in-context spatial concatenation in the latent space. It concatenates images horizontally as input, using a (target | blank) configuration for generation tasks and (target | source) for editing tasks. To stabilize the training of this compact model, we introduce a task-progressive joint pretraining strategy that sequentially targets T2I, editing, and joint tasks. After high-quality SFT and reinforcement learning, DreamLite achieves GenEval (0.72) for image generation and ImgEdit (4.11) for image editing, outperforming existing on-device models and remaining competitive with several server-side models. By employing step distillation, we further reduce denoising processing to just 4 steps, enabling our DreamLite could generate or edit a 1024 x 1024 image in less than 1s on a Xiaomi 14 smartphone. To the best of our knowledge, DreamLite is the first unified on-device diffusion model that supports both image generation and image editing.
Related papers
- BlazeEdit: Generalist Image Editing on Mobile Devices with Image-to-Image Diffusion Models [13.439538133009018]
BlazeEdit is a highly efficient, generalist image-to-image diffusion model tailored for on-device deployment.<n>BlazeEdit achieves a substantial reduction in download size and memory overhead while maintaining competitive generation quality.<n>It completes a full inference pass in just 290ms on a Pixel 10, delivering a seamless, privacy-preserving, and lightning-fast experience.
arXiv Detail & Related papers (2026-05-27T07:23:59Z) - HiDream-O1-Image: A Natively Unified Image Generative Foundation Model with Pixel-level Unified Transformer [104.09730595701468]
We present HiDream-O1-Image, a unified generative foundation model via pixel-space Diffusion Transformer.<n>By mapping raw image pixels, text tokens, and task-specific conditions into a single shared token space, HiDream-O1-Image achieves a structural unification of multimodal inputs.<n>Experiments show that HiDream-O1-Image excels across various generation tasks, including text-to-image generation, instruction-based editing, and subject-driven personalization.
arXiv Detail & Related papers (2026-05-11T17:59:09Z) - Seedream 4.0: Toward Next-generation Multimodal Image Generation [88.86697995940511]
Seedream 4.0 is an efficient and high-performance multimodal image generation system.<n>It unifies text-to-image (T2I) synthesis, image editing, and multi-image composition within a single framework.<n>Seedream 4.0 is pretrained on billions of text-image pairs spanning diverse and knowledge-centric concepts.
arXiv Detail & Related papers (2025-09-24T17:59:04Z) - DreamOmni: Unified Image Generation and Editing [76.46811926046225]
We introduce Dream Omni, a unified model for image generation and editing.<n>For training, Dream Omni jointly trains T2I generation and downstream tasks.<n>This collaboration significantly boosts editing performance.
arXiv Detail & Related papers (2024-12-22T17:17:28Z) - SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training [77.681908636429]
Text-to-image (T2I) models face several limitations, including large model sizes, slow, and low-quality generation on mobile devices.<n>This paper aims to develop an extremely small and fast T2I model that generates high-resolution and high-quality images on mobile platforms.
arXiv Detail & Related papers (2024-12-12T18:59:53Z) - DreamSync: Aligning Text-to-Image Generation with Image Understanding
Feedback [38.81701138951801]
Text-to-Image models (T2I) still struggle to produce images that are both aesthetically pleasing and faithful to the user's input text.
We introduce DreamSync, a model-agnostic training algorithm by design that improves T2I models to be faithful to the text input.
Despite its simplicity, DreamSync improves both the semantic alignment and aesthetic appeal of two diffusion-based T2I models.
arXiv Detail & Related papers (2023-11-29T03:42:16Z) - SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two
Seconds [88.06788636008051]
Text-to-image diffusion models can create stunning images from natural language descriptions that rival the work of professional artists and photographers.
These models are large, with complex network architectures and tens of denoising iterations, making them computationally expensive and slow to run.
We present a generic approach that unlocks running text-to-image diffusion models on mobile devices in less than $2$ seconds.
arXiv Detail & Related papers (2023-06-01T17:59:25Z) - GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation [143.81719619351335]
Text-to-image (T2I) models based on diffusion processes have achieved remarkable success in controllable image generation using user-provided captions.
The tight coupling between the current text encoder and image decoder in T2I models makes it challenging to replace or upgrade.
We propose GlueGen, which applies a newly proposed GlueNet model to align features from single-modal or multi-modal encoders with the latent space of an existing T2I model.
arXiv Detail & Related papers (2023-03-17T15:37:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.