DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval

• URL: http://arxiv.org/abs/2603.09185v1
• Date: Tue, 10 Mar 2026 04:47:15 GMT
• Title: DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval
• Abstract summary: We propose Direct Embedding Optimization (DEO), a training-free method for negation-aware text and multimodal retrieval.<n>DEO decomposes queries into positive and negative components and optimize the query embedding with a contrastive objective.<n>Without additional training data or model updates, DEO outperforms baselines on NegConstraint, with gains of +0.0738 nDCG@10 and +0.1028 MAP@100.<n>These results demonstrate the practicality of DEO for negation- and exclusion-aware retrieval in real-world settings.
• Score: 6.8984945362190695
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) have enabled diverse retrieval methods. However, existing retrieval methods often fail to accurately retrieve results for negation and exclusion queries. To address this limitation, prior approaches rely on embedding adaptation or fine-tuning, which introduce additional computational cost and deployment complexity. We propose Direct Embedding Optimization (DEO), a training-free method for negation-aware text and multimodal retrieval. DEO decomposes queries into positive and negative components and optimizes the query embedding with a contrastive objective. Without additional training data or model updates, DEO outperforms baselines on NegConstraint, with gains of +0.0738 nDCG@10 and +0.1028 MAP@100, while improving Recall@5 by +6\% over OpenAI CLIP in multimodal retrieval. These results demonstrate the practicality of DEO for negation- and exclusion-aware retrieval in real-world settings.

Related papers

• LLM-Guided Evolutionary Search for Constraint Model Reformulation to Improve Solver Efficiency [5.774119507201688]
  Large Language Models (LLMs) can produce correct models from natural language, but even a correct model can be expensive to solve.<n>We use an evolutionary framework in which an LLM proposes candidate reformulations that are verified and benchmarked against the user-defined baseline model.<n>Existing retention strategies prioritize recency or performance, but do not explicitly diversify the context.<n>We systematically evaluate the strategies on eight CSPLib problems using validation-based final model selection.
  arXiv  Detail & Related papers  (2026-07-30T14:21:55Z)
• F-GRPO: Factorized Group-Relative Policy Optimization for Unified Candidate Generation and Ranking [79.49893545611779]
  Large Language Models (LLMs) can generate a subset and order it within a single autoregressive pass.<n>This flexibility introduces a new optimization challenge: the model must search an output space while receiving utility feedback only after the full ranked list is generated.<n>This credit assignment gap makes end-to-end optimization unstable and sample-inefficient.<n>We propose a unified framework that performs both within a single autoregressive rollout and optimize them end-to-end via factorized group-relative policy optimization.
  arXiv  Detail & Related papers  (2026-05-13T04:52:33Z)
• Reproducing Complex Set-Compositional Information Retrieval [11.891887979573925]
  Complex information needs may involve set-compositional queries using conjunction, disjunction, and exclusion.<n>We benchmark major retrieval families and reasoning-targeted methods on QUEST and QUEST+Variants.
  arXiv  Detail & Related papers  (2026-05-05T14:51:40Z)
• Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models? [65.18157595903124]
  This work investigates iterative approximate evaluation for arbitrary prompts.<n>It introduces Model Predictive Prompt Selection (MoPPS), a Bayesian risk-predictive framework.<n>MoPPS reliably predicts prompt difficulty and accelerates training with significantly reduced rollouts.
  arXiv  Detail & Related papers  (2025-07-07T03:20:52Z)
• Maximally-Informative Retrieval for State Space Model Generation [59.954191072042526]
  We introduce Retrieval In-Context Optimization (RICO) to minimize model uncertainty for a particular query at test-time.<n>Unlike traditional retrieval-augmented generation (RAG), which relies on externals for document retrieval, our approach leverages direct feedback from the model.<n>We show that standard top-$k$ retrieval with model gradients can approximate our optimization procedure, and provide connections to the leave-one-out loss.
  arXiv  Detail & Related papers  (2025-06-13T18:08:54Z)
• Explicit Preference Optimization: No Need for an Implicit Reward Model [18.225409932618657]
  Direct preference optimization (DPO) and its offshoots circumvent the need for a separate reward training step.<n>We show that DPO-based objectives are nonetheless subject to sub-optimal regularization and counter-intuitive artifacts.
  arXiv  Detail & Related papers  (2025-06-09T07:11:01Z)
• Embed Progressive Implicit Preference in Unified Space for Deep Collaborative Filtering [13.24227546548424]
  Generalized Neural Ordinal Logistic Regression (GNOLR) is proposed to capture the structured progression of user engagement.<n>GNOLR enhances predictive accuracy, captures the progression of user engagement, and simplifies the retrieval process.<n>Experiments on ten real-world datasets show that GNOLR significantly outperforms state-of-the-art methods in efficiency and adaptability.
  arXiv  Detail & Related papers  (2025-05-27T08:43:35Z)
• Optimizing Retrieval Augmented Generation for Object Constraint Language [3.4777703321218225]
  OCL is essential for Model-Based Systems Engineering (MBSE) but manually writing OCL rules is complex and time-consuming.<n>We evaluate the impact of three different retrieval strategies on $OCLBERT generation.<n>We show that while retrieval can enhance generation accuracy, its effectiveness depends on the retrieval method and the number of retrieved chunks.
  arXiv  Detail & Related papers  (2025-05-19T14:00:10Z)
• In-context Demonstration Matters: On Prompt Optimization for Pseudo-Supervision Refinement [71.60563181678323]
  Large language models (LLMs) have achieved great success across diverse tasks, and fine-tuning is sometimes needed to further enhance generation quality.<n>To handle these challenges, a direct solution is to generate high-confidence'' data from unsupervised downstream tasks.<n>We propose a novel approach, pseudo-supervised demonstrations aligned prompt optimization (PAPO) algorithm, which jointly refines both the prompt and the overall pseudo-supervision.
  arXiv  Detail & Related papers  (2024-10-04T03:39:28Z)
• Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment [104.18002641195442]
  We introduce Self-Augmented Preference Optimization (SAPO), an effective and scalable training paradigm that does not require existing paired data. Building on the self-play concept, which autonomously generates negative responses, we further incorporate an off-policy learning pipeline to enhance data exploration and exploitation.
  arXiv  Detail & Related papers  (2024-05-31T14:21:04Z)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [62.824464372594576]
  We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization. We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
  arXiv  Detail & Related papers  (2023-09-13T01:12:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.