VaultGemma: A Differentially Private Gemma Model

• URL: http://arxiv.org/abs/2510.15001v2
• Date: Wed, 22 Oct 2025 23:11:16 GMT
• Title: VaultGemma: A Differentially Private Gemma Model
• Abstract summary: We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy.<n>We openly release this model to the community.
• Score: 80.35976343610584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy. Pretrained on the identical data mixture used for the Gemma 2 series, VaultGemma 1B represents a significant step forward in privacy-preserving large language models. We openly release this model to the community

Related papers

• Gemma 4 Technical Report [199.38174392182012]
  We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family.<n>The Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters.<n>We propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches.
  arXiv  Detail & Related papers  (2026-07-02T21:08:53Z)
• Silhouette-based Gait Foundation Model [56.27974816297294]
  Building a unified gait foundation model requires addressing two longstanding barriers: Scalability and Generalization.<n>We introduce FoundationGait, the first scalable, self-supervised pretraining framework for gait understanding.
  arXiv  Detail & Related papers  (2025-11-30T01:53:41Z)
• Gemma 3 Technical Report [198.3299202423321]
  We introduce Gemma 3, a multimodal addition to the Gemma family of lightweight open models.<n>This version introduces vision understanding abilities, a wider coverage of languages and longer context.<n>We also change the architecture of the model to reduce the KV-cache memory that tends to explode with long context.
  arXiv  Detail & Related papers  (2025-03-25T15:52:34Z)
• BgGPT 1.0: Extending English-centric LLMs to other languages [12.867025651644692]
  We present BgGPT-Gemma-2-27B-Instruct and BgGPT-Gemma-2-9B-Instruct: continually pretrained and fine-tuned versions of Google's Gemma-2 models.<n>Our models demonstrate strong performance in Bulgarian language tasks, setting a new standard for language-specific AI models.
  arXiv  Detail & Related papers  (2024-12-14T16:49:52Z)
• Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 [11.169778211035826]
  Sparse autoencoders (SAEs) are an unsupervised method for learning a neural network's latent representations into seemingly interpretable features. In this work, we introduce Gemma Scope, an open suite of JumpReLU SAEs trained on all layers and sub-layers of Gemma 2 2B and 9B. We evaluate the quality of each SAE on standard metrics and release these results.
  arXiv  Detail & Related papers  (2024-08-09T16:06:42Z)
• Gemma 2: Improving Open Language Models at a Practical Size [118.04200128754249]
  We introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models. We apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger.
  arXiv  Detail & Related papers  (2024-07-31T19:13:07Z)
• RecurrentGemma: Moving Past Transformers for Efficient Open Language Models [103.59785165735727]
  We introduce RecurrentGemma, a family of open language models using Google's novel Griffin architecture. Griffin combines linear recurrences with local attention to achieve excellent performance on language. We provide two sizes of models, containing 2B and 9B parameters, and provide pre-trained and instruction tuned variants for both.
  arXiv  Detail & Related papers  (2024-04-11T15:27:22Z)
• Gemma: Open Models Based on Gemini Research and Technology [128.57714343844074]
  This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety.
  arXiv  Detail & Related papers  (2024-03-13T06:59:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.