PC-MNet: Dual-Level Congruity Modeling for Multimodal Sarcasm Detection via Polarity-Modulated Attention

• URL: http://arxiv.org/abs/2605.02447v1
• Date: Mon, 04 May 2026 10:47:17 GMT
• Title: PC-MNet: Dual-Level Congruity Modeling for Multimodal Sarcasm Detection via Polarity-Modulated Attention
• Abstract summary: This work provides a robust, decoupled paradigm for modeling subtle pragmatic incongruities in human communication.<n>Experiments on the textttMUStARD benchmark and its spurious-correlation-mitigated balanced datasets demonstrate that our approach achieves new state-of-the-art performance.
• Score: 32.62063298936575
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal sarcasm detection, which aims to precisely identify pragmatic incongruities between literal text and nonverbal cues, has gained substantial attention in multimodal understanding. Recent advancements have predominantly relied on naïve similarity-based attention mechanisms and uniform late fusion strategies.Furthermore, given that functional entanglement restricts traditional late fusions, we incorporate a scalar congruity routing mechanism and a prior-guided contextual graph. This mechanism anchors a generalized incongruity manifold through a two-stage asymmetric optimization driven by inconsistency-aware contrastive learning, selectively fusing only the most discriminative multi-granularity evidence. Extensive experiments on the \texttt{MUStARD} benchmark and its spurious-correlation-mitigated balanced datasets demonstrate that our approach achieves new state-of-the-art performance, surpassing the strongest multimodal baseline by a substantial 3.14\% improvement in Macro-F1. By architecturally isolating atomic, composition, and contextual conflicts. This work provides a robust, decoupled paradigm for modeling subtle pragmatic incongruities in human communication.

Related papers

• Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection [8.747820204756566]
  Multimodal sarcasm detection aims to identify sarcastic intent from multimodal content.<n>New framework integrates Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization.
  arXiv  Detail & Related papers  (2026-08-20T12:02:06Z)
• URMF: Uncertainty-aware Robust Multimodal Fusion for Multimodal Sarcasm Detection [17.19666367498091]
  Multimodal sarcasm detection (MSD) aims to identify sarcastic intent from semantic incongruity between text and image.<n>We propose Uncertainty-aware Robust Multimodal Fusion (URMF), a unified framework that explicitly models modality reliability during interaction and fusion.<n>URMF consistently outperforms strong unimodal, multimodal, and MLLM-based baselines.
  arXiv  Detail & Related papers  (2026-04-08T06:50:43Z)
• Feature-level Interaction Explanations in Multimodal Transformers [1.7101146971136896]
  Multimodal Transformers produce predictions without clarifying how different modalities jointly support a decision.<n>We present Feature-level I2MoE, a structured Mixture-of-Experts layer that operates directly on token/patch sequences from frozen pretrained encoders.<n>We develop an expert-wise explanation pipeline that combines attribution with top-K% masking to assess faithfulness.
  arXiv  Detail & Related papers  (2026-03-04T18:24:31Z)
• Together, Then Apart: Revisiting Multimodal Survival Analysis via a Min-Max Perspective [22.583594870571336]
  This work revisits multi-modal survival analysis via the dual lens of alignment and distinctiveness.<n>We introduce Together-Then-Apart, a unified min-max optimization framework that simultaneously models shared and modality-specific representations.<n>Our formulation provides a new theoretical perspective of how alignment and distinctiveness can be jointly achieved in for robust, interpretable, and biologically meaningful multi-modal survival analysis.
  arXiv  Detail & Related papers  (2025-11-22T15:10:46Z)
• Multiview Manifold Evidential Fusion for PolSAR Image Classification [51.41332458376411]
  We propose a new framework to integrate PolSAR manifold learning and evidence fusion into a unified architecture.<n>Experiments on three real-world PolSAR datasets demonstrate that the proposed method consistently outperforms existing approaches in accuracy, robustness, and interpretability.
  arXiv  Detail & Related papers  (2025-10-13T09:05:51Z)
• Harnessing Consistency for Robust Test-Time LLM Ensemble [88.55393815158608]
  CoRE is a plug-and-play technique that harnesses model consistency for robust LLM ensemble.<n> Token-level consistency captures fine-grained disagreements by applying a low-pass filter to downweight uncertain tokens.<n>Model-level consistency models global agreement by promoting model outputs with high self-confidence.
  arXiv  Detail & Related papers  (2025-10-12T04:18:45Z)
• Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning [49.17801010041155]
  Multimodal large language models (MLLMs) promise enhanced reasoning by integrating diverse inputs such as text, vision, and audio.<n>Yet cross-modal reasoning remains underexplored, with conflicting reports on whether added modalities help or harm performance.<n>We categorize multimodal reasoning into six interaction patterns, varying how facts are distributed across modalities and logically combined.
  arXiv  Detail & Related papers  (2025-09-28T08:46:11Z)
• FindRec: Stein-Guided Entropic Flow for Multi-Modal Sequential Recommendation [57.577843653775]
  We propose textbfFindRec (textbfFlexible unified textbfinformation textbfdisentanglement for multi-modal sequential textbfRecommendation)<n>A Stein kernel-based Integrated Information Coordination Module (IICM) theoretically guarantees distribution consistency between multimodal features and ID streams.<n>A cross-modal expert routing mechanism that adaptively filters and combines multimodal features based on their contextual relevance.
  arXiv  Detail & Related papers  (2025-07-07T04:09:45Z)
• Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations [19.731611716111566]
  We propose a Multimodal fusion approach for learning modality-Exclusive and modality-Agnostic representations. We introduce a predictive self-attention module to capture reliable context dynamics within modalities. A hierarchical cross-modal attention module is designed to explore valuable element correlations among modalities. A double-discriminator strategy is presented to ensure the production of distinct representations in an adversarial manner.
  arXiv  Detail & Related papers  (2024-07-06T04:36:48Z)
• Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition [69.32305810128994]
  Incongruity between modalities poses a challenge for multimodal fusion, especially in affect recognition. We propose the Hierarchical Crossmodal Transformer with Dynamic Modality Gating (HCT-DMG), a lightweight incongruity-aware model. HCT-DMG: 1) outperforms previous multimodal models with a reduced size of approximately 0.8M parameters; 2) recognizes hard samples where incongruity makes affect recognition difficult; 3) mitigates the incongruity at the latent level in crossmodal attention.
  arXiv  Detail & Related papers  (2023-05-23T01:24:15Z)
• Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis [96.46952672172021]
  Bi-Bimodal Fusion Network (BBFN) is a novel end-to-end network that performs fusion on pairwise modality representations. Model takes two bimodal pairs as input due to known information imbalance among modalities.
  arXiv  Detail & Related papers  (2021-07-28T23:33:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.