Solution to the 10th ABAW Expression Recognition Challenge: A Robust Multimodal Framework with Safe Cross-Attention and Modality Dropout

• URL: http://arxiv.org/abs/2603.08034v1
• Date: Mon, 09 Mar 2026 07:13:20 GMT
• Title: Solution to the 10th ABAW Expression Recognition Challenge: A Robust Multimodal Framework with Safe Cross-Attention and Modality Dropout
• Abstract summary: We propose a multimodal framework that dynamically fuses visual and audio representations.<n>Our approach uses a dual-branch Transformer architecture featuring a safe cross-attention mechanism and a modality dropout strategy.<n>This design allows the network to rely on audio-based predictions when visual cues are absent.
• Score: 20.96260280449496
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Emotion recognition in real-world environments is hindered by partial occlusions, missing modalities, and severe class imbalance. To address these issues, particularly for the Affective Behavior Analysis in-the-wild (ABAW) Expression challenge, we propose a multimodal framework that dynamically fuses visual and audio representations. Our approach uses a dual-branch Transformer architecture featuring a safe cross-attention mechanism and a modality dropout strategy. This design allows the network to rely on audio-based predictions when visual cues are absent. To mitigate the long-tail distribution of the Aff-Wild2 dataset, we apply focal loss optimization, combined with a sliding-window soft voting strategy to capture dynamic emotional transitions and reduce frame-level classification jitter. Experiments demonstrate that our framework effectively handles missing modalities and complex spatiotemporal dependencies, achieving an accuracy of 60.79% and an F1-score of 0.5029 on the Aff-Wild2 validation set.

Related papers

• Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration [7.647944993041615]
  We introduce a training-free inference strategy to dynamically correct semantic drift without relying on abrupt truncations.<n>By computing an outlier-resistant statistical midpoint across various attention heads, we establish a stable anchor for reliable visual representations.<n>This recalibration process effectively rectifies visual-semantic misalignments while fully preserving generative fluency and language priors.
  arXiv  Detail & Related papers  (2026-05-24T09:14:49Z)
• Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approach [53.64487844936037]
  Our method combines three complementary modalities: face, behavior, and audio.<n>Experiments demonstrate that the proposed multimodal fusion strategy achieves a Concordance Correlation Coefficient (CCC) of 0.658 on the Aff-Wild2 development set.
  arXiv  Detail & Related papers  (2026-03-13T15:06:14Z)
• A Two-Stage Dual-Modality Model for Facial Emotional Expression Recognition [5.014305646574725]
  This paper addresses the expression (EXPR) recognition challenge in the 10th Affective Behavior Analysis in-the-Wild (ABAW) workshop and competition.<n>It requires frame-level classification of eight facial emotional expressions from unconstrained videos.<n>We propose a two-stage dual-modal (audio-visual) model to address these difficulties.
  arXiv  Detail & Related papers  (2026-03-12T17:45:12Z)
• Multimodal Emotion Recognition via Bi-directional Cross-Attention and Temporal Modeling [2.8037951156321377]
  We propose a multimodal emotion recognition framework for the Expression task in the 10th Affective Behavior Analysis in-the-wild (ABAW) Challenge.<n>Our framework builds on large-scale pre-trained models for visual and audio representation learning and integrates them in a unified multimodal architecture.<n> Experimental results on the ABAW 10th EXPR benchmark demonstrate the effectiveness of the proposed framework.
  arXiv  Detail & Related papers  (2026-03-12T14:20:29Z)
• Hierarchical Granularity Alignment and State Space Modeling for Robust Multimodal AU Detection in the Wild [12.893641381621459]
  We propose a novel multimodal framework driven by Hierarchical Granularity Alignment and State Space Models.<n>We overcome the receptive field limitations of conventional temporal convolutional networks by introducing a Vision-Mamba architecture.<n>This approach enables temporal modeling with O(N) linear complexity, effectively capturing ultra-long-range dynamics without performance degradation.
  arXiv  Detail & Related papers  (2026-03-11T21:05:59Z)
• OmniVL-Guard: Towards Unified Vision-Language Forgery Detection and Grounding via Balanced RL [63.388513841293616]
  Existing forgery detection methods fail to handle the interleaved text, images, and videos prevalent in real-world misinformation.<n>To bridge this gap, this paper targets to develop a unified framework for omnibus vision-language forgery detection and grounding.<n>We propose textbf OmniVL-Guard, a balanced reinforcement learning framework for omnibus vision-language forgery detection and grounding.
  arXiv  Detail & Related papers  (2026-02-11T09:41:36Z)
• Synergistic Prompting for Robust Visual Recognition with Missing Modalities [13.821274074204082]
  Large-scale multi-modal models have demonstrated remarkable performance across various visual recognition tasks.<n>The presence of missing or incomplete modality inputs often leads to significant performance degradation.<n>We propose a novel Synergistic Prompting framework for robust visual recognition with missing modalities.
  arXiv  Detail & Related papers  (2025-07-10T14:28:12Z)
• UGG-ReID: Uncertainty-Guided Graph Model for Multi-Modal Object Re-Identification [26.770271366177603]
  We propose a robust approach named Uncertainty-Guided Graph model for multi-modal object ReID (UGG-ReID)<n>UGG-ReID is designed to mitigate noise interference and facilitate effective multi-modal fusion.<n> Experimental results show that the proposed method achieves excellent performance on all datasets.
  arXiv  Detail & Related papers  (2025-07-07T03:41:08Z)
• AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection [57.649223695021114]
  We present a novel weakly supervised framework that leverages audio-visual collaboration for robust video anomaly detection.<n>Our framework demonstrates superior performance across multiple benchmarks, with audio integration significantly boosting anomaly detection accuracy.
  arXiv  Detail & Related papers  (2025-04-06T13:59:16Z)
• Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval [7.439049772394586]
  Diffusion Augmented Retrieval (DAR) is a framework that generates multiple intermediate representations via dialogue refinements and DMs.<n>DAR results on par with finetuned I-TIR models, yet without incurring their tuning overhead.
  arXiv  Detail & Related papers  (2025-01-26T03:29:18Z)
• Low-Light Video Enhancement via Spatial-Temporal Consistent Decomposition [52.89441679581216]
  Low-Light Video Enhancement (LLVE) seeks to restore dynamic or static scenes plagued by severe invisibility and noise.<n>We present an innovative video decomposition strategy that incorporates view-independent and view-dependent components.<n>Our framework consistently outperforms existing methods, establishing a new SOTA performance.
  arXiv  Detail & Related papers  (2024-05-24T15:56:40Z)
• A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition [53.800937914403654]
  Advanced Audio-Visual Speech Recognition (AVSR) systems have been observed to be sensitive to missing video frames. While applying the dropout technique to the video modality enhances robustness to missing frames, it simultaneously results in a performance loss when dealing with complete data input. We propose a novel Multimodal Distribution Approximation with Knowledge Distillation (MDA-KD) framework to reduce over-reliance on the audio modality.
  arXiv  Detail & Related papers  (2024-03-07T06:06:55Z)
• NR-DFERNet: Noise-Robust Network for Dynamic Facial Expression Recognition [1.8604727699812171]
  We propose a noise-robust dynamic facial expression recognition network (NR-DFERNet) to reduce the interference of noisy frames on the DFER task. Specifically, at the spatial stage, we devise a dynamic-static fusion module (DSF) that introduces dynamic features to static features for learning more discriminative spatial features. To suppress the impact of target irrelevant frames, we introduce a novel dynamic class token (DCT) for the transformer at the temporal stage.
  arXiv  Detail & Related papers  (2022-06-10T10:17:30Z)
• Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification [208.1227090864602]
  Visible-infrared person re-identification (VI-ReID) is a challenging cross-modality pedestrian retrieval problem. Existing VI-ReID methods tend to learn global representations, which have limited discriminability and weak robustness to noisy images. We propose a novel dynamic dual-attentive aggregation (DDAG) learning method by mining both intra-modality part-level and cross-modality graph-level contextual cues for VI-ReID.
  arXiv  Detail & Related papers  (2020-07-18T03:08:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.