Learning to Detect Cross-Modal Negation: An Analysis of Latent Representations and an Attention-Based Solution

• URL: http://arxiv.org/abs/2607.17712v1
• Date: Mon, 20 Jul 2026 09:08:51 GMT
• Title: Learning to Detect Cross-Modal Negation: An Analysis of Latent Representations and an Attention-Based Solution
• Abstract summary: We show that negation does not form a linearly or non-linearly separable class in the latent spaces of standard vision-language models.<n>We propose a novel cross-modal attention architecture that explicitly models inter-modal dependencies.
• Score: 44.39196719594625
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting high-level semantic concepts like negation across modalities remains a challenge for current multimodal systems. We analyze this as a fundamental representation learning problem, providing the first evidence that negation does not form a linearly or non-linearly separable class in the latent spaces of standard vision-language models (VLMs). We demonstrate that pretrained embeddings primarily encode modality-specific features, lacking a generalizable negation signal. To overcome this, we propose a novel cross-modal attention architecture that explicitly models inter-modal dependencies, achieving performance gains of up to +7.03% F1 over unimodal baselines. Our analysis reveals a key asymmetry: while textual negation often appears independently, visual negation is semantically dependent on linguistic context, a finding validated through our statistical analysis of 3,222 political video-text pairs automatically annotated via \textsc{Qwen2.5-VL}. By combining this analysis with self-supervised video representations (JEPA2), we advance the modeling of temporal negation. This work provides new methods and insights for learning robust, semantically-aligned representations in multimodal systems.

Related papers

• Revisiting the Systematicity in Negation in the Era of In-Context Learning [16.384578665588936]
  We analyze systematicity regarding understanding of negation from two perspectives: behavioral systematicity and representational systematicity.<n>For behavioral systematicity, we confirm that through demonstrations and in-context learning, LLMs can recognize negation expressions and scope within sentences to some extent, but they fail to achieve perfect performance.<n>For representational systematicity, we analyze the extent to which function vectors can be robustly constructed from in-context examples for tasks that are essential to understanding negation.
  arXiv  Detail & Related papers  (2026-06-15T15:44:15Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• Exposing the Unsaid: Visualizing Hidden LLM Bias through Stochastic Path Aggregation [16.63373250752545]
  Large Language Models (LLMs) exhibit representational and syntactic biases that are difficult to evaluate due to the marginalization of text generation.<n>This paper introduces TreeTracer, a visual analytics tool designed to evaluate bias through aggregated comparison.
  arXiv  Detail & Related papers  (2026-04-24T13:37:51Z)
• When Negation Is a Geometry Problem in Vision-Language Models [32.51815690470519]
  Joint Vision-Language Embedding models such as CLIP typically fail at understanding negation in text queries.<n>We investigate an alternative evaluation framework based on Multimodal LLMs-as-a-judge, which excel at understanding simple yes/no questions about image content.
  arXiv  Detail & Related papers  (2026-03-20T23:06:23Z)
• Mastering Negation: Boosting Grounding Models via Grouped Opposition-Based Learning [87.15765427638195]
  We introduce D-Negation, a new dataset that provides objects annotated with both positive and negative semantic descriptions.<n>We propose a grouped opposition-based learning framework that learns negation-aware representations from limited samples.<n>By fine-tuning fewer than 10 percent of the model parameters, our approach achieves improvements of up to 4.4 mAP and 5.7 mAP on positive and negative semantic evaluations.
  arXiv  Detail & Related papers  (2026-03-13T03:21:48Z)
• Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities? [61.533560295383786]
  Unified Multimodal Large Language Models (U-MLLMs) integrate understanding and generation within a single architecture.<n>We observe that U-MLLMs fail to maintain semantic equivalence when required to render the same results in the image modality.<n>We introduce VGUBench, a framework to decouple reasoning logic from generation fidelity.
  arXiv  Detail & Related papers  (2026-02-27T06:23:56Z)
• Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations [34.88156871518115]
  Next-token prediction (NTP) optimization leads language models to extract and organize semantic structure from text.<n>We demonstrate that concepts corresponding to larger singular values are learned earlier during training, yielding a natural semantic hierarchy.<n>This insight motivates orthant-based clustering, a method that combines concept signs to identify interpretable semantic categories.
  arXiv  Detail & Related papers  (2025-05-13T08:46:04Z)
• A Novel Energy based Model Mechanism for Multi-modal Aspect-Based Sentiment Analysis [85.77557381023617]
  We propose a novel framework called DQPSA for multi-modal sentiment analysis. PDQ module uses the prompt as both a visual query and a language query to extract prompt-aware visual information. EPE module models the boundaries pairing of the analysis target from the perspective of an Energy-based Model.
  arXiv  Detail & Related papers  (2023-12-13T12:00:46Z)
• Improving negation detection with negation-focused pre-training [58.32362243122714]
  Negation is a common linguistic feature that is crucial in many language understanding tasks. Recent work has shown that state-of-the-art NLP models underperform on samples containing negation. We propose a new negation-focused pre-training strategy, involving targeted data augmentation and negation masking.
  arXiv  Detail & Related papers  (2022-05-09T02:41:11Z)
• Language Models for Lexical Inference in Context [4.581468205348204]
  Lexical inference in context (LIiC) is the task of recognizing textual entailment between two very similar sentences. We formulate and evaluate the first approaches based on pretrained language models (LMs) for this task. All our approaches outperform the previous state of the art, showing the potential of pretrained LMs for LIiC.
  arXiv  Detail & Related papers  (2021-02-10T09:08:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.