Point-Supervised Facial Expression Spotting with Gaussian-Based Instance-Adaptive Intensity Modeling

• URL: http://arxiv.org/abs/2511.16952v2
• Date: Wed, 26 Nov 2025 06:01:16 GMT
• Title: Point-Supervised Facial Expression Spotting with Gaussian-Based Instance-Adaptive Intensity Modeling
• Abstract summary: We propose a two-branch framework for point-supervised facial expression spotting (P-FES)<n>First, to mitigate the limitation of hard pseudo-labeling, we propose a Gaussian-based instance-adaptive intensity modeling (GIM) module.<n>Second, we design a class-aware apex classification branch that distinguishes macro- and micro-expressions solely based on their pseudo-apex frames.
• Score: 18.418576406317538
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Automatic facial expression spotting, which aims to identify facial expression instances in untrimmed videos, is crucial for facial expression analysis. Existing methods primarily focus on fully-supervised learning and rely on costly, time-consuming temporal boundary annotations. In this paper, we investigate point-supervised facial expression spotting (P-FES), where only a single timestamp annotation per instance is required for training. We propose a unique two-branch framework for P-FES. First, to mitigate the limitation of hard pseudo-labeling, which often confuses neutral and expression frames with various intensities, we propose a Gaussian-based instance-adaptive intensity modeling (GIM) module to model instance-level expression intensity distribution for soft pseudo-labeling. By detecting the pseudo-apex frame around each point label, estimating the duration, and constructing an instance-level Gaussian distribution, GIM assigns soft pseudo-labels to expression frames for more reliable intensity supervision. The GIM module is incorporated into our framework to optimize the class-agnostic expression intensity branch. Second, we design a class-aware apex classification branch that distinguishes macro- and micro-expressions solely based on their pseudo-apex frames. During inference, the two branches work independently: the class-agnostic expression intensity branch generates expression proposals, while the class-aware apex-classification branch is responsible for macro- and micro-expression classification. Furthermore, we introduce an intensity-aware contrastive loss to enhance discriminative feature learning and suppress neutral noise by contrasting neutral frames with expression frames with various intensities. Extensive experiments on the SAMM-LV, CAS(ME)$^2$, and CAS(ME)$^3$ datasets demonstrate the effectiveness of our proposed framework.

Related papers

• AnchorMoE: Interpretable Time Series Classification via Anchor-Routed MoE [51.994752158953084]
  AnchorMoE is an interpretable-by-construction classification framework.<n>It encodes multi-view representations of local patches and routes them to specialized experts.<n>Experiments on real-world and synthetic benchmarks demonstrate that AnchorMoE achieves highly competitive classification performance.
  arXiv  Detail & Related papers  (2026-06-02T13:30:54Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processes [60.75226150503949]
  We propose a Bayesian framework that quantifies semantic uncertainty by analyzing the geometric structure of answer embeddings.<n>S GPU maps generated answers into a dense semantic space, computes the Gram matrix of their semantic embeddings, and summarizes their semantic configuration.<n>We show that S GPU transfers across models and modalities, indicating that its spectral representation captures general patterns of semantic uncertainty.
  arXiv  Detail & Related papers  (2025-12-16T08:15:24Z)
• Weakly Supervised Continuous Micro-Expression Intensity Estimation Using Temporal Deep Neural Network [0.0]
  We propose a unified framework for continuous micro-expression intensity estimation using only weak temporal labels.<n>A simple triangular prior converts sparse temporal landmarks into dense pseudo-intensity trajectories.<n>Experiments on SAMM and CASME II show strong temporal agreement with the pseudo-intensity trajectories.
  arXiv  Detail & Related papers  (2025-11-30T23:47:47Z)
• Robust Dynamic Facial Expression Recognition [6.626374248579249]
  This paper proposes a robust method of distinguishing between hard and noisy samples.<n>To identify the principal expression in a video, a key expression re-sampling framework and a dual-stream hierarchical network is proposed.<n>The proposed method has been shown to outperform current State-Of-The-Art approaches in DFER.
  arXiv  Detail & Related papers  (2025-02-22T07:48:12Z)
• LEAF: Unveiling Two Sides of the Same Coin in Semi-supervised Facial Expression Recognition [56.22672276092373]
  Semi-supervised learning has emerged as a promising approach to tackle the challenge of label scarcity in facial expression recognition.<n>We propose a unified framework termed hierarchicaL dEcoupling And Fusing (LEAF) to coordinate expression-relevant representations and pseudo-labels for semi-supervised FER.
  arXiv  Detail & Related papers  (2024-04-23T13:43:33Z)
• Weak Supervision with Arbitrary Single Frame for Micro- and Macro-expression Spotting [22.04975008531069]
  We propose a point-level weakly-supervised expression spotting framework, where each expression requires to be annotated with only one random frame (i.e., a point) We show MPLG generates more reliable pseudo labels by merging class-specific probabilities, attention scores, fused features, and point-level labels. Experiments on the CAS(ME)2, CAS(ME)3, and SAMM-LV datasets demonstrate PWES achieves promising performance comparable to that of recent fully-supervised methods.
  arXiv  Detail & Related papers  (2024-03-21T09:01:21Z)
• Weakly-supervised Micro- and Macro-expression Spotting Based on Multi-level Consistency [22.7160073059238]
  Weakly-supervised expression spotting (WES) based on video-level labels can potentially mitigate the complexity of frame-level annotation. We propose a novel and simple WES framework, MC-WES, using multi-consistency collaborative mechanisms. We show that MC-WES is comparable to state-of-the-art fully-supervised methods.
  arXiv  Detail & Related papers  (2023-05-04T11:14:47Z)
• MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection [76.80153360498797]
  We develop a multiple instance self-training framework (MIST) to efficiently refine task-specific discriminative representations. MIST is composed of 1) a multiple instance pseudo label generator, which adapts a sparse continuous sampling strategy to produce more reliable clip-level pseudo labels, and 2) a self-guided attention boosted feature encoder. Our method performs comparably to or even better than existing supervised and weakly supervised methods, specifically obtaining a frame-level AUC 94.83% on ShanghaiTech.
  arXiv  Detail & Related papers  (2021-04-04T15:47:14Z)
• Semi-supervised Facial Action Unit Intensity Estimation with Contrastive Learning [54.90704746573636]
  Our method does not require to manually select key frames, and produces state-of-the-art results with as little as $2%$ of annotated frames. We experimentally validate that our method outperforms existing methods when working with as little as $2%$ of randomly chosen data.
  arXiv  Detail & Related papers  (2020-11-03T17:35:57Z)
• Mutual Information Regularized Identity-aware Facial ExpressionRecognition in Compressed Video [27.602648102881535]
  We propose a novel collaborative min-min game for mutual information (MI) minimization in latent space. We do not need the identity label or multiple expression samples from the same person for identity elimination. Our solution can achieve comparable or better performance than the recent decoded image-based methods.
  arXiv  Detail & Related papers  (2020-10-20T21:42:18Z)
• Causal Intervention for Weakly-Supervised Semantic Segmentation [122.1846968696862]
  We aim to generate better pixel-level pseudo-masks by using only image-level labels. We propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification.
  arXiv  Detail & Related papers  (2020-09-26T09:26:29Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.