EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters

• URL: http://arxiv.org/abs/2402.04252v1
• Date: Tue, 6 Feb 2024 18:59:48 GMT
• Title: EVA-CLIP-18B: Scaling CLIP to 18 Billion Parameters
• Abstract summary: We present EVA-CLIP-18B, the largest and most powerful open-source CLIP model to date with 18-billion parameters. With only 6-billion training samples seen, EVA-CLIP-18B achieves an exceptional 80.7% zero-shot top-1 accuracy averaged across 27 widely recognized image classification benchmarks.
• Score: 25.729577042823514
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scaling up contrastive language-image pretraining (CLIP) is critical for empowering both vision and multimodal models. We present EVA-CLIP-18B, the largest and most powerful open-source CLIP model to date, with 18-billion parameters. With only 6-billion training samples seen, EVA-CLIP-18B achieves an exceptional 80.7% zero-shot top-1 accuracy averaged across 27 widely recognized image classification benchmarks, outperforming its forerunner EVA-CLIP (5-billion parameters) and other open-source CLIP models by a large margin. Remarkably, we observe a consistent performance improvement with the model size scaling of EVA-CLIP, despite maintaining a constant training dataset of 2-billion image-text pairs from LAION-2B and COYO-700M. This dataset is openly available and much smaller than the in-house datasets (e.g., DFN-5B, WebLI-10B) employed in other state-of-the-art CLIP models. EVA-CLIP-18B demonstrates the potential of EVA-style weak-to-strong visual model scaling. With our model weights made publicly available, we hope to facilitate future research in vision and multimodal foundation models.

Related papers

• Fine-tuning RoBERTa for CVE-to-CWE Classification: A 125M Parameter Model Competitive with LLMs [0.0]
  We construct a large-scale training dataset of 234,770 descriptions with AI-refined CWE labels.<n>On our held-out test set (27,780 samples, 205 CWE classes), the model achieves 87.4% top-1 accuracy.<n>On the external CTI-Bench benchmark (NeurIPS 2024), the model achieves 75.6% strict accuracy.
  arXiv  Detail & Related papers  (2026-03-16T07:15:57Z)
• VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling [60.341503853471494]
  We show that vision-language-action models degrade sharply under novel camera viewpoints and visual perturbations.<n>We propose a one-shot adaptation framework that recalibrates visual representations through lightweight, learnable updates.
  arXiv  Detail & Related papers  (2025-12-02T16:16:13Z)
• Evo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment [13.582337741042773]
  Vision-Language-Action (VLA) models have emerged as a powerful framework that unifies perception, language, and control, enabling robots to perform diverse tasks through multimodal understanding.<n>Current VLA models typically contain massive parameters and rely heavily on large-scale robot data pretraining, leading to high computational costs during training, as well as limited deployability for real-time inference.<n>We present Evo-1, a lightweight VLA model that reduces computation and improves deployment efficiency, while maintaining strong performance without pretraining on robot data.
  arXiv  Detail & Related papers  (2025-11-06T17:07:49Z)
• Trade-offs in Cross-Domain Generalization of Foundation Model Fine-Tuned for Biometric Applications [18.08946802592489]
  Foundation models such as CLIP have demonstrated exceptional zero- and few-shot transfer capabilities across diverse vision tasks.<n>However, fine-tuned for highly specialized biometric tasks, face recognition (FR), morphing attack detection (MAD), and presentation attack detection (PAD) may suffer from over-specialization.<n>We systematically quantify these trade-offs by evaluating three instances of CLIP fine-tuned for FR, MAD, and PAD.
  arXiv  Detail & Related papers  (2025-09-18T12:58:18Z)
• Accessing Vision Foundation Models via ImageNet-1K [51.521125501182816]
  Proteus is trained at ImageNet-level costs with surprising ability, facilitating the accessibility of training foundation models for the broader research community. Proteus-L/14 matches the performance of the Oracle method DINOv2-L/14 across 19 benchmarks and outperforms other vision foundation models including CLIP-L/14 (400M), OpenCLIP-L/14 (400M/2B) and SynCLR-L/14 (600M) with a significantly smaller training set of 1.2M images.
  arXiv  Detail & Related papers  (2024-07-15T00:13:53Z)
• OpenVLA: An Open-Source Vision-Language-Action Model [131.74098076670103]
  We introduce OpenVLA, an open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA shows strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate. We release model checkpoints, fine-tuning notebooks, and our PyTorch with built-in support for training VLAs at scale on Open X-Embodiment datasets.
  arXiv  Detail & Related papers  (2024-06-13T15:46:55Z)
• RLAIF-V: Aligning MLLMs through Open-Source AI Feedback for Super GPT-4V Trustworthiness [94.03511733306296]
  We introduce RLAIF-V, a framework that aligns MLLMs in a fully open-source paradigm for super GPT-4V trustworthiness. RLAIF-V maximally exploits the open-source feedback from two perspectives, including high-quality feedback data and online feedback learning algorithm. Experiments show that RLAIF-V substantially enhances the trustworthiness of models without sacrificing performance on other tasks.
  arXiv  Detail & Related papers  (2024-05-27T14:37:01Z)
• ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generations [67.25974711647481]
  Text-to-image (T2I) diffusion models, notably the unCLIP models, achieve state-of-the-art (SOTA) performance on various compositional T2I benchmarks. We introduce ECLIPSE, a novel contrastive learning method that is both parameter and data-efficient. We demonstrate that ECLIPSE trained prior, with only 3.3% of the parameters and trained on a mere 2.8% of the data, surpasses the baseline T2I priors with an average of 71.6% preference score.
  arXiv  Detail & Related papers  (2023-12-07T19:32:39Z)
• MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training [17.158498267947877]
  We introduce MobileCLIP, a new family of efficient image-text models optimized for runtime performance. MobileCLIP uses knowledge transfer from an image captioning model and an ensemble of strong CLIP encoders to improve the accuracy of efficient models. Our approach avoids train-time compute overhead by storing the additional knowledge in a reinforced dataset.
  arXiv  Detail & Related papers  (2023-11-28T18:55:42Z)
• EVA-CLIP: Improved Training Techniques for CLIP at Scale [20.145062325090286]
  We propose EVA-CLIP, a series of models that significantly improve the efficiency and effectiveness of CLIP training. Our approach incorporates new techniques for representation learning, optimization, and augmentation, enabling EVA-CLIP to achieve superior performance.
  arXiv  Detail & Related papers  (2023-03-27T17:02:21Z)
• EVA-02: A Visual Representation for Neon Genesis [49.90565085768437]
  EVA-02 is a next-generation Transformer-based visual representation pre-trained to reconstruct strong and robust language-aligned vision features. We offer four EVA-02 variants in various model sizes, ranging from 6M to 304M parameters, all with impressive performance.
  arXiv  Detail & Related papers  (2023-03-20T17:59:59Z)
• Face Recognition in the age of CLIP & Billion image datasets [0.0]
  We evaluate the performance of various CLIP models as zero-shot face recognizers. We also investigate the robustness of CLIP models against data poisoning attacks.
  arXiv  Detail & Related papers  (2023-01-18T05:34:57Z)
• Rethinking Mobile Block for Efficient Attention-based Models [60.0312591342016]
  This paper focuses on developing modern, efficient, lightweight models for dense predictions while trading off parameters, FLOPs, and performance. Inverted Residual Block (IRB) serves as the infrastructure for lightweight CNNs, but no counterpart has been recognized by attention-based studies. We extend CNN-based IRB to attention-based models and abstracting a one-residual Meta Mobile Block (MMB) for lightweight model design.
  arXiv  Detail & Related papers  (2023-01-03T15:11:41Z)
• ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond [76.35955924137986]
  We propose a Vision Transformer Advanced by Exploring intrinsic IB from convolutions, i.e., ViTAE. ViTAE has several spatial pyramid reduction modules to downsample and embed the input image into tokens with rich multi-scale context. We obtain the state-of-the-art classification performance, i.e., 88.5% Top-1 classification accuracy on ImageNet validation set and the best 91.2% Top-1 accuracy on ImageNet real validation set.
  arXiv  Detail & Related papers  (2022-02-21T10:40:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.