Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models

• URL: http://arxiv.org/abs/2605.21573v1
• Date: Wed, 20 May 2026 17:59:58 GMT
• Title: Lens: Rethinking Training Efficiency for Foundational Text-to-Image Models
• Abstract summary: We introduce Lens, a T2I model that achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6B parameters.<n>For example, Lens requires only about 19.3% of the training compute used by Z-Image.<n>Lens generates a 10242 image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds.
• Score: 88.94987156920575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce Lens, a 3.8B-parameter T2I model that achieves performance competitive with, and in several cases surpassing, state-of-the-art models with more than 6B parameters across various benchmarks, while requiring significantly less training compute. For example, Lens requires only about 19.3% of the training compute used by Z-Image. The training efficiency of Lens stems from two key strategies beyond its compact model size. First, we maximize data information density per training batch by (i) training on Lens-800M, a dataset of 800M densely captioned image-text pairs whose captions are generated by GPT-4.1 and contain approximately 109 words on average, providing richer semantic supervision than conventional short captions, and (ii) constructing each batch from images with multiple resolutions and diverse aspect ratios, thereby enlarging the effective visual coverage of each optimization step. Second, we improve convergence speed through careful architectural choices, including adopting a semantic VAE that provides better latent representations and employing a strong language encoder that accelerates optimization while enabling multilingual generalization from English-only training data. After pre-training, we apply RL with taxonomy-driven prompts (Lens-RL-8K) and structured reward rubrics to suppress artifacts and improve visual quality, a reasoner module with training-free system prompt search to better align user requests with the model, and distillation-based acceleration for 4-step inference. Through efficient training and systematic optimization, Lens generalizes to arbitrary aspect ratios from 1:2 to 2:1 and resolutions up to 1440^2, and supports prompts in several commonly used languages. Thanks to its compact size, Lens generates a 1024^2 image in 3.15 seconds on a single NVIDIA H100 GPU, while its distilled turbo version performs 4-step generation in 0.84 seconds.

Related papers

• Nucleus-Image: Sparse MoE for Image Generation [5.769753912757775]
  We present a text-to-image generation model that exceeds leading models on GenEval, DPG-Bench, and OneIG-Bench.<n>Nucleus-Image employs a sparse mixture-of-experts (MoE) diffusion transformer architecture.<n>We construct a large-scale training corpus of 1.5B high-quality training pairs spanning 700M images.
  arXiv  Detail & Related papers  (2026-04-14T00:43:23Z)
• TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge [23.707347449137895]
  TernaryCLIP is a lightweight framework that converts connection weights of both vision and text encoders of CLIP into the ternary format.<n>Our work highlights the feasibility of extreme quantization for large multimodal models, supporting effective and efficient deployment on resource-constrained devices.
  arXiv  Detail & Related papers  (2025-10-23T14:53:32Z)
• Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignment [67.94226713255732]
  We propose a dual-branch offset learning paradigm that learns feature and class offsets to dynamically refine both class representations and spatial image features.<n>Based on the proposed paradigm, we construct an efficient semantic segmentation network, OffSeg.<n>Experiments on four datasets, including ADE20K, Cityscapes, COCO-Stuff-164K, and Pascal Context, demonstrate consistent improvements with negligible parameters.
  arXiv  Detail & Related papers  (2025-08-12T10:10:10Z)
• Analysis of Hyperparameter Optimization Effects on Lightweight Deep Models for Real-Time Image Classification [0.0]
  This study evaluates the accuracy and deployment feasibility of seven modern lightweight architectures: ConvNeXt-T, EfficientV2-S, MobileNetV3-L, MobileViT v2 (S/XS), RepVGG-A2, and TinyViT-21M.<n> tuning alone leads to a top-1 accuracy improvement of 1.5 to 3.5 percent over baselines, and select models deliver latency under 5 milliseconds and over 9,800 frames per second.
  arXiv  Detail & Related papers  (2025-07-31T07:47:30Z)
• FG-CLIP: Fine-Grained Visual and Textual Alignment [3.830067625507938]
  We propose Fine-Grained CLIP, which enhances fine-grained understanding through three key innovations.<n>We leverage large multimodal models to generate 1.6 billion long caption-image pairs for capturing global-level semantic details.<n>We construct a comprehensive dataset, termed FineHARD, by integrating high-quality region-specific annotations with hard fine-grained negative samples.
  arXiv  Detail & Related papers  (2025-05-08T09:06:53Z)
• Pretrained Image-Text Models are Secretly Video Captioners [38.66202065611397]
  We find that an image-based model can be repurposed to outperform several specialised video captioning systems.<n>Our adapted model demonstrates top tier performance on major benchmarks, ranking 2nd on MSRVTT and MSVD, and 3rd on VATEX.<n>From a resource optimization perspective, this video captioning study focuses on three fundamental factors: optimizing model scale, maximizing data efficiency, and incorporating reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-19T01:53:03Z)
• Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models [111.97026994761254]
  Mixture-of-Transformers (MoT) is a sparse multi-modal transformer architecture.<n>MoT decouples non-embedding parameters of the model by modality.<n>We evaluate MoT across multiple settings and model scales.
  arXiv  Detail & Related papers  (2024-11-07T18:59:06Z)
• UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks [60.46473247205654]
  Using large-scale unsupervised unimodal models as pre-training can enhance the zero-shot performance of image-text pair models. Our experiments show that unimodal pre-training outperforms state-of-the-art CLIP-based models.
  arXiv  Detail & Related papers  (2023-06-07T18:26:22Z)
• Joint Adaptive Representations for Image-Language Learning [59.40890927221377]
  We propose a recipe for image-language learning, which produces effective models, outperforming bigger and more expensive ones, often trained on orders of magnitude larger datasets. Our key finding is the joint learning of a compact vision and language representation, which adaptively and iteratively fuses the multi-modal features. With only 40M training examples and with 39 GFLOPs our lightweight model outperforms many times larger state-of-the-art models of 2-20x more FLOPs and using bigger datasets some of which with close to 1B training examples.
  arXiv  Detail & Related papers  (2023-05-31T15:02:02Z)
• EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning [19.354515754130592]
  We introduce a distilling then pruning framework to compress large vision-language models into smaller, faster, and more accurate ones. We apply our framework to train EfficientVLM, a fast and accurate vision-language model consisting of 6 vision layers, 3 text layers, and 3 cross-modal fusion layers. EfficientVLM retains 98.4% performance of the teacher model and accelerates its inference speed by 2.2x.
  arXiv  Detail & Related papers  (2022-10-14T13:26:41Z)
• Expanding Language-Image Pretrained Models for General Video Recognition [136.0948049010682]
  Contrastive language-image pretraining has shown great success in learning visual-textual joint representation from web-scale data. We present a simple yet effective approach that adapts the pretrained language-image models to video recognition directly. Our approach surpasses the current state-of-the-art methods by +7.6% and +14.9% in terms of top-1 accuracy under two popular protocols.
  arXiv  Detail & Related papers  (2022-08-04T17:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.