Micro-AU CLIP: Fine-Grained Contrastive Learning from Local Independence to Global Dependency for Micro-Expression Action Unit Detection

• URL: http://arxiv.org/abs/2603.16302v1
• Date: Tue, 17 Mar 2026 09:38:04 GMT
• Title: Micro-AU CLIP: Fine-Grained Contrastive Learning from Local Independence to Global Dependency for Micro-Expression Action Unit Detection
• Abstract summary: This paper proposes a novel micro-AU detection framework, micro-AU CLIP, that decomposes the AU detection process into local semantic independence modeling (LSI) and global semantic dependency (GSD) modeling.<n>The experimental results demonstrate that Micro-AU CLIP can fully learn fine-grained micro-AU features, achieving state-of-the-art performance.
• Score: 22.850797928500153
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Micro-expression (ME) action units (Micro-AUs) provide objective clues for fine-grained genuine emotion analysis. Most existing Micro-AU detection methods learn AU features from the whole facial image/video, which conflicts with the inherent locality of AU, resulting in insufficient perception of AU regions. In fact, each AU independently corresponds to specific localized facial muscle movements (local independence), while there is an inherent dependency between some AUs under specific emotional states (global dependency). Thus, this paper explores the effectiveness of the independence-to-dependency pattern and proposes a novel micro-AU detection framework, micro-AU CLIP, that uniquely decomposes the AU detection process into local semantic independence modeling (LSI) and global semantic dependency (GSD) modeling. In LSI, Patch Token Attention (PTA) is designed, mapping several local features within the AU region to the same feature space; In GSD, Global Dependency Attention (GDA) and Global Dependency Loss (GDLoss) are presented to model the global dependency relationships between different AUs, thereby enhancing each AU feature. Furthermore, considering CLIP's native limitations in micro-semantic alignment, a microAU contrastive loss (MiAUCL) is designed to learn AU features by a fine-grained alignment of visual and text features. Also, Micro-AU CLIP is effectively applied to ME recognition in an emotion-label-free way. The experimental results demonstrate that Micro-AU CLIP can fully learn fine-grained micro-AU features, achieving state-of-the-art performance.

Related papers

• Micro-Defects Expose Macro-Fakes: Detecting AI-Generated Images via Local Distributional Shifts [48.75898711300078]
  generative models can produce images that appear highly realistic, raising challenges in distinguishing real and AI-generated images.<n>We propose Micro-Defects expose Macro-Fakes (MDMF), a local distribution-aware detection framework that amplifies micro-scale statistical irregularities into macro-level distributional discrepancies.<n>Our theory-grounded analysis shows that patch-wise modeling yields provably larger discrepancies when localized forensic signals are present in generated images.
  arXiv  Detail & Related papers  (2026-05-10T03:44:09Z)
• Zero-shot HOI Detection with MLLM-based Detector-agnostic Interaction Recognition [71.5328300638085]
  Zero-shot Human-object interaction (HOI) detection aims to locate humans and objects in images and recognize their interactions.<n>Existing methods, including two-stage methods, tightly couple interaction recognition with a specific detector.<n>We propose a decoupled framework that separates object detection from IR and leverages multi-modal large language models (MLLMs) for zero-shot IR.
  arXiv  Detail & Related papers  (2026-02-16T19:01:31Z)
• AU-LLM: Micro-Expression Action Unit Detection via Enhanced LLM-Based Feature Fusion [26.058143518505805]
  This paper introduces textbfAU-LLM, a novel framework that uses Large Language Models to detect micro-expression Action Units (AUs) in micro-expression datasets with subtle intensities and the scarcity of data.<n>We specifically address the critical vision-language semantic gap, the textbfEnhanced Fusion Projector (EFP). The EFP employs a Multi-Layer Perceptron (MLP) to intelligently fuse mid-level (local texture) and high-level (global semantics) visual features from a specialized 3D-CNN backbone into a single, information-dense token.
  arXiv  Detail & Related papers  (2025-07-29T13:01:59Z)
• Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample [53.23474626420103]
  Facial action unit (AU) detection remains a challenging task, due to the subtlety, dynamics, and diversity of AUs. We propose a novel AU detection framework called AC2D by adaptively constraining self-attention weight distribution. Our method achieves competitive performance compared to state-of-the-art AU detection approaches on challenging benchmarks.
  arXiv  Detail & Related papers  (2024-10-02T05:51:24Z)
• Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning [48.70249675019288]
  We propose an end-to-end Vision-Language joint learning network for explainable facial action units (AUs) recognition. The proposed approach achieves superior performance over the state-of-the-art methods on most metrics.
  arXiv  Detail & Related papers  (2024-08-01T15:35:44Z)
• Adaptive Global-Local Representation Learning and Selection for Cross-Domain Facial Expression Recognition [54.334773598942775]
  Domain shift poses a significant challenge in Cross-Domain Facial Expression Recognition (CD-FER) We propose an Adaptive Global-Local Representation Learning and Selection framework.
  arXiv  Detail & Related papers  (2024-01-20T02:21:41Z)
• Self-supervised Facial Action Unit Detection with Region and Relation Learning [5.182661263082065]
  We propose a novel self-supervised framework for AU detection with the region and relation learning. An improved Optimal Transport (OT) algorithm is introduced to exploit the correlation characteristics among AUs. Swin Transformer is exploited to model the long-distance dependencies within each AU region during feature learning.
  arXiv  Detail & Related papers  (2023-03-10T05:22:45Z)
• Local Attention Graph-based Transformer for Multi-target Genetic Alteration Prediction [0.22940141855172028]
  We propose a general-purpose local attention graph-based Transformer for MIL (LA-MIL) We demonstrate that LA-MIL achieves state-of-the-art results in mutation prediction for gastrointestinal cancer. This suggests that local self-attention sufficiently models dependencies on par with global modules.
  arXiv  Detail & Related papers  (2022-05-13T14:24:24Z)
• MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection [16.261362598190807]
  The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images. We argue that encoding AU features just from one perspective may not capture the rich contextual information between regional and global face features. We propose a novel Multi-level Graph Reasoning Network (termed MGRR-Net) for facial AU detection.
  arXiv  Detail & Related papers  (2022-04-04T09:47:22Z)
• Objective Class-based Micro-Expression Recognition through Simultaneous Action Unit Detection and Feature Aggregation [18.35953886595087]
  We propose a novel deep neural network model for objective class-based Micro-Expression Recognition (MER) Our model simultaneously detects Action Units (AUs) and aggregates AU-level features into micro-expression-level representation. Our approach significantly outperforms the current state-of-the-arts in MER.
  arXiv  Detail & Related papers  (2020-12-24T07:31:15Z)
• J$\hat{\text{A}}$A-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention [57.51255553918323]
  We propose a novel end-to-end deep learning framework for joint AU detection and face alignment. Our framework significantly outperforms the state-of-the-art AU detection methods on the challenging BP4D, DISFA, GFT and BP4D+ benchmarks.
  arXiv  Detail & Related papers  (2020-03-18T12:50:19Z)
• Facial Action Unit Detection via Adaptive Attention and Relation [40.34933431651346]
  We propose a novel adaptive attention and relation (AAR) framework for facial AU detection. Specifically, we propose an adaptive attention regression network to regress the global attention map of each AU under the constraint of attention predefinition. Considering the diversity and dynamics of AUs, we propose an adaptive-temporal graphal network to simultaneously reason independent pattern of each AU, the inter-dependencies among AUs, as well as the temporal dependencies.
  arXiv  Detail & Related papers  (2020-01-05T05:14:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.