MEME-Fusion@CHiPSAL 2026: Multimodal Ablation Study of Hate Detection and Sentiment Analysis on Nepali Memes

• URL: http://arxiv.org/abs/2604.14218v1
• Date: Mon, 13 Apr 2026 07:37:14 GMT
• Title: MEME-Fusion@CHiPSAL 2026: Multimodal Ablation Study of Hate Detection and Sentiment Analysis on Nepali Memes
• Abstract summary: This paper presents our system for the CHiPSAL 2026 shared task, addressing both Subtask A and Subtask B.<n>We propose a hybrid cross-modal attention fusion architecture that combines CLIP for visual encoding with BGE-M3 for multilingual text representation.<n>We show explicit cross-modal reasoning achieves a 5.9% F1-macro improvement over text-only baselines on Subtask A.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hate speech detection in Devanagari-scripted social media memes presents compounded challenges: multimodal content structure, script-specific linguistic complexity, and extreme data scarcity in low-resource settings. This paper presents our system for the CHiPSAL 2026 shared task, addressing both Subtask A (binary hate speech detection) and Subtask B (three-class sentiment classification: positive, neutral, negative). We propose a hybrid cross-modal attention fusion architecture that combines CLIP (ViT-B/32) for visual encoding with BGE-M3 for multilingual text representation, connected through 4-head self-attention and a learnable gating network that dynamically weights modality contributions on a per-sample basis. Systematic evaluation across eight model configurations demonstrates that explicit cross-modal reasoning achieves a 5.9% F1-macro improvement over text-only baselines on Subtask A, while uncovering two unexpected but critical findings: English-centric vision models exhibit near-random performance on Devanagari script, and standard ensemble methods catastrophically degrade under data scarcity (N nearly equal to 850 per fold) due to correlated overfitting. The code can be accessed at https://github.com/Tri-Yantra-Technologies/MEME-Fusion/

Related papers

• MVFA: A Multi-View Text-Guided Multimodal Fusion LLM Adapter for Sentiment Analysis and Emotion Recognition [104.40097283944003]
  We propose a multi-view text-guided multimodal fusion adapter (MVFA) for affective computing.<n>MVFA is evaluated on three challenging datasets: CH-SIMS V2.0, MELD, and CHERMA.<n>It achieves state-of-the-art performance on key metrics while updating only a small fraction of parameters.
  arXiv  Detail & Related papers  (2026-09-05T17:12:51Z)
• Team RAS in 11th ABAW Competition: Multimodal Ambivalence Recognition Approach [52.92973028971813]
  We present a single text-centered multimodal approach for video-level ambivalence and hesitancy recognition.<n>The proposed approach combines linguistic, acoustic, facial, and scene features using text-centered multimodal fusion model.<n>Experiments on the Behavioural Ambivalence/Hesitancy (BAH) corpus confirm that text is the strongest unimodal modality.
  arXiv  Detail & Related papers  (2026-07-16T08:04:23Z)
• Bilingual Text-to-Motion Generation: A New Benchmark and Baselines [52.71312720094036]
  We introduce BiHumanML3D, the first bilingual text-to-motion benchmark, constructed via LLM-assisted annotation and rigorous manual correction.<n>We also propose a simple yet effective baseline, Bilingual Motion Diffusion synthesis (BiMD), featuring Cross-Lingual Alignment (CLA)<n>BiMD with CLA achieves an FID of 0.045 vs. 0.169 and R@3 of 82.8% vs. 80.8%, significantly outperforms monolingual diffusion models and translation baselines on BiHumanML3D.
  arXiv  Detail & Related papers  (2026-03-26T08:48:27Z)
• Labels or Input? Rethinking Augmentation in Multimodal Hate Detection [9.166963162285064]
  We present a dual-pronged approach to improve multimodal hate detection.<n>First, we propose a prompt optimization framework that systematically varies prompt structure, supervision, and training modality.<n>Second, we introduce a multimodal data augmentation pipeline that generates 2,479 counterfactually neutral memes.
  arXiv  Detail & Related papers  (2025-08-15T21:31:00Z)
• Team "better_call_claude": Style Change Detection using a Sequential Sentence Pair Classifier [5.720553544629197]
  At PAN 2025, the shared task challenges participants to detect style at the most fine-grained level: individual sentences.<n>We propose to address this problem by modeling the content of each instance using a Sentence Pair Pair (SSPC) architecture.<n>The model achieves strong macro macro scores of 0.92328, and 0.724 on the EASY MEDIUM, and HARD data, respectively.
  arXiv  Detail & Related papers  (2025-08-01T14:48:17Z)
• Knowing Where to Focus: Attention-Guided Alignment for Text-based Person Search [64.15205542003056]
  We introduce Attention-Guided Alignment (AGA) framework featuring two innovative components: Attention-Guided Mask (AGM) Modeling and Text Enrichment Module (TEM)<n>AGA achieves new state-of-the-art results with Rank-1 accuracy reaching 78.36%, 67.31%, and 67.4% on CUHK-PEDES, ICFG-PEDES, and RSTP, respectively.
  arXiv  Detail & Related papers  (2024-12-19T17:51:49Z)
• A Unified Multi-Task Learning Architecture for Hate Detection Leveraging User-Based Information [23.017068553977982]
  Hate speech, offensive language, aggression, racism, sexism, and other abusive language are common phenomena in social media. There is a need for Artificial Intelligence(AI)based intervention which can filter hate content at scale. This paper introduces a unique model that improves hate speech identification for the English language by utilising intra-user and inter-user-based information.
  arXiv  Detail & Related papers  (2024-11-11T10:37:11Z)
• Align before Attend: Aligning Visual and Textual Features for Multimodal Hateful Content Detection [4.997673761305336]
  This paper proposes a context-aware attention framework for multimodal hateful content detection. We evaluate the proposed approach on two benchmark hateful meme datasets, viz. MUTE (Bengali code-mixed) and MultiOFF (English)
  arXiv  Detail & Related papers  (2024-02-15T06:34:15Z)
• Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023 [51.95161901441527]
  In this paper, we propose a novel framework for recognizing both discrete and dimensional emotions. Deep features extracted from foundation models are used as robust acoustic and visual representations of raw video. Our final system achieves state-of-the-art performance and ranks third on the leaderboard on MER-MULTI sub-challenge.
  arXiv  Detail & Related papers  (2023-09-11T03:19:10Z)
• Noisy-Correspondence Learning for Text-to-Image Person Re-identification [50.07634676709067]
  We propose a novel Robust Dual Embedding method (RDE) to learn robust visual-semantic associations even with noisy correspondences. Our method achieves state-of-the-art results both with and without synthetic noisy correspondences on three datasets.
  arXiv  Detail & Related papers  (2023-08-19T05:34:13Z)
• Multi-Grained Multimodal Interaction Network for Entity Linking [65.30260033700338]
  Multimodal entity linking task aims at resolving ambiguous mentions to a multimodal knowledge graph. We propose a novel Multi-GraIned Multimodal InteraCtion Network $textbf(MIMIC)$ framework for solving the MEL task.
  arXiv  Detail & Related papers  (2023-07-19T02:11:19Z)
• Exploiting BERT For Multimodal Target SentimentClassification Through Input Space Translation [75.82110684355979]
  We introduce a two-stream model that translates images in input space using an object-aware transformer. We then leverage the translation to construct an auxiliary sentence that provides multimodal information to a language model. We achieve state-of-the-art performance on two multimodal Twitter datasets.
  arXiv  Detail & Related papers  (2021-08-03T18:02:38Z)
• A study of text representations in Hate Speech Detection [0.0]
  Current EU and US legislation against hateful language has led to automatic tools being a necessary component of the Hate Speech detection task and pipeline. In this study, we examine the performance of several, diverse text representation techniques paired with multiple classification algorithms, on the automatic Hate Speech detection task.
  arXiv  Detail & Related papers  (2021-02-08T20:39:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.