jina-embeddings-v5-omni: Geometry-preserving Embeddings via Locked Aligned Towers

• URL: http://arxiv.org/abs/2605.08384v2
• Date: Tue, 12 May 2026 17:52:45 GMT
• Title: jina-embeddings-v5-omni: Geometry-preserving Embeddings via Locked Aligned Towers
• Abstract summary: GELATO is a novel approach to multimodal embedding models.<n>GELATO produces results that are competitive with the state-of-the-art.
• Score: 4.3450882150663235
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this work, we introduce GELATO (Geometry-preserving Embeddings via Locked Aligned TOwers), a novel approach to multimodal embedding models. We build on the VLM-style architecture, in which non-text encoders are adapted to produce input for a language model, which in turn generates embeddings for all varieties of input. We present the result: the jina-embeddings-v5-omni suite, a pair of models that encode text, image, audio, and video input into a single semantic embedding space. GELATO extends the two Jina Embeddings v5 Text models to support additional modality by adding encoders for images and audio. The backbone text embedding models and the added non-text modality encoders remain frozen. We only trained the connecting components, representing 0.35% of the total weights of the joint model. Training is therefore much more efficient than full-parameter retraining. Additionally, the language model remains effectively unaltered, producing exactly the same embeddings for text inputs as the Jina Embeddings v5 Text models. Our evaluations show that GELATO produces results that are competitive with the state-of-the-art, yielding nearly equal performance to larger multimodal embedding models.

Related papers

• jina-embeddings-v5-text: Task-Targeted Embedding Distillation [4.215793601372204]
  General-purpose models are typically trained with single- or multi-stage processes using contrastive loss functions.<n>We introduce a novel training regimen that combines model distillation techniques with task-specific contrastive loss to produce compact embedding models.<n> Benchmark scores for the resulting models exceed or match the state-of-the-art for models of similar size.
  arXiv  Detail & Related papers  (2026-02-17T12:50:50Z)
• EmbeddingGemma: Powerful and Lightweight Text Representations [42.34520055523387]
  EmbeddingGemma is a new lightweight, open text embedding model based on the Gemma 3 language family.<n>We improve model robustness and expressiveness with a spread-out regularizer.<n>We release EmbeddingGemma to the community to promote further research.
  arXiv  Detail & Related papers  (2025-09-24T17:56:51Z)
• Text-to-Level Diffusion Models With Various Text Encoders for Super Mario Bros [0.0]
  We present strategies to automatically assign captions to an existing dataset, and train diffusion models using both pretrained text encoders and simple transformer models trained from scratch.<n>Results are compared with an unconditional diffusion model and a generative adversarial network, as well as the text-to-level approaches Five-Dollar Model and MarioGPT.
  arXiv  Detail & Related papers  (2025-06-30T18:50:26Z)
• The Best of Both Worlds: Integrating Language Models and Diffusion Models for Video Generation [53.837937703425794]
  LanDiff is a hybrid framework that synergizes the strengths of autoregressive language models and diffusion models.<n>Our architecture introduces three key innovations: (1) a semantic tokenizer that compresses 3D visual features into compact 1D representations through efficient semantic compression; (2) a language model that generates semantic tokens with high-level semantic relationships; and (3) a streaming diffusion model that refines coarse semantics into high-fidelity videos.
  arXiv  Detail & Related papers  (2025-03-06T16:53:14Z)
• Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think [38.258453761376586]
  We propose Dream Engine, an efficient framework designed for arbitrary text-image interleaved control in image generation models.<n>Our approach utilizes a two-stage training paradigm, consisting of joint text-image alignment and multimodal interleaved instruction tuning.<n>Our experiments demonstrate that this training method is effective, achieving a 0.69 overall score on the GenEval benchmark.
  arXiv  Detail & Related papers  (2025-02-27T15:08:39Z)
• ChatGarment: Garment Estimation, Generation and Editing via Large Language Models [79.46056192947924]
  ChatGarment is a novel approach that leverages large vision-language models (VLMs) to automate the estimation, generation, and editing of 3D garments.<n>It can estimate sewing patterns from in-the-wild images or sketches, generate them from text descriptions, and edit garments based on user instructions.
  arXiv  Detail & Related papers  (2024-12-23T18:59:28Z)
• Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models [5.2094499417507105]
  This report describes the training dataset creation and recipe behind the family of textttarctic-embed text embedding models. At the time of their release, each model achieved state-of-the-art retrieval accuracy for models of their size on the MTEB Retrieval leaderboard.
  arXiv  Detail & Related papers  (2024-05-08T19:05:18Z)
• Multilingual E5 Text Embeddings: A Technical Report [63.503320030117145]
  Three embedding models of different sizes are provided, offering a balance between the inference efficiency and embedding quality. We introduce a new instruction-tuned embedding model, whose performance is on par with state-of-the-art, English-only models of similar sizes.
  arXiv  Detail & Related papers  (2024-02-08T13:47:50Z)
• TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering [118.30923824681642]
  TextDiffuser-2 aims to unleash the power of language models for text rendering. We utilize the language model within the diffusion model to encode the position and texts at the line level. We conduct extensive experiments and incorporate user studies involving human participants as well as GPT-4V.
  arXiv  Detail & Related papers  (2023-11-28T04:02:40Z)
• The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddings [3.620115940532283]
  The five-dollar model is a lightweight text-to-image generative architecture that generates low dimensional images from an encoded text prompt. We apply this model to three small datasets: pixel art video game maps, video game sprite images, and down-scaled emoji images. We evaluate our models performance using cosine similarity score between text-image pairs generated by the CLIP VIT-B/32 model.
  arXiv  Detail & Related papers  (2023-08-08T05:16:51Z)
• Generating Images with Multimodal Language Models [78.6660334861137]
  We propose a method to fuse frozen text-only large language models with pre-trained image encoder and decoder models. Our model demonstrates a wide suite of multimodal capabilities: image retrieval, novel image generation, and multimodal dialogue.
  arXiv  Detail & Related papers  (2023-05-26T19:22:03Z)
• eDiffi: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers [87.52504764677226]
  Large-scale diffusion-based generative models have led to breakthroughs in text-conditioned high-resolution image synthesis. We train an ensemble of text-to-image diffusion models specialized for different stages synthesis. Our ensemble of diffusion models, called eDiffi, results in improved text alignment while maintaining the same inference cost.
  arXiv  Detail & Related papers  (2022-11-02T17:43:04Z)
• Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization [108.09419317477986]
  Z-Code++ is a new pre-trained language model optimized for abstractive text summarization. The model is first pre-trained using text corpora for language understanding, and then is continually pre-trained on summarization corpora for grounded text generation. Our model is parameter-efficient in that it outperforms the 600x larger PaLM-540B on XSum, and the finetuned 200x larger GPT3-175B on SAMSum.
  arXiv  Detail & Related papers  (2022-08-21T01:00:54Z)
• Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval [70.30052749168013]
  Multi-channel video-language retrieval require models to understand information from different channels. contrastive multimodal models are shown to be highly effective at aligning entities in images/videos and text. There is not a clear way to quickly adapt these two lines to multi-channel video-language retrieval with limited data and resources.
  arXiv  Detail & Related papers  (2022-06-05T01:43:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.