Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras

• URL: http://arxiv.org/abs/2609.06928v1
• Date: Mon, 07 Sep 2026 01:55:44 GMT
• Title: Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras
• Abstract summary: Bio-inspired event cameras capture brightness changes, thereby reducing exposure of facial identity details.<n>We propose the Information-Guided Gated Fusion (IGF) framework, which pre-trains an event encoder on the FAU subset of Emo-DVS to capture fine-grained facial dynamics.<n>To alleviate data scarcity, we introduce Emo-DVS, the first large-scale event-based emotion analysis dataset.
• Score: 28.949201907906765
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Emotion analysis is a fundamental task in computer vision, but its practical deployment remains constrained by the privacy risks inherent to conventional RGB cameras. Bio-inspired event cameras present a promising hardware-level solution because they capture asynchronous brightness changes, thereby reducing exposure of facial identity details while leveraging high dynamic range for robust perception under challenging illumination conditions. Despite these advantages, existing event-based methods struggle in complex real-world settings due to limited dataset scales, simple acquisition conditions, and reliance on single-modality visual cues. To address these, we establish a challenging tri-modal benchmark with event, audio, and text modalities and propose the Information-Guided Gated Fusion (IGF) framework, which first pre-trains an event encoder on the FAU subset of Emo-DVS to capture fine-grained facial dynamics, then employs adaptive modality gating to suppress modality-specific noise, and finally leverages mutual information maximization to align robust cross-modal representations. To alleviate data scarcity, we introduce Emo-DVS, the first large-scale event-based emotion analysis dataset, which couples dynamic illumination with the Facial Action Unit (FAU) subset and emotion subset. Extensive experiments demonstrate that IGF achieves state-of-the-art performance.

Related papers

• Event-Illumination Collaborative Low-light Image Enhancement with a High-resolution Real-world Dataset [74.66758770936579]
  EIC-LIE is an event-illumination collaborative LIE framework.<n>EIC-LIE outperforms state-of-the-art methods on five real-world and synthetic datasets.
  arXiv  Detail & Related papers  (2026-05-21T08:55:34Z)
• EventGait: Towards Robust Gait Recognition with Event Streams [50.890621860023]
  Event cameras offer microsecond temporal resolution and high dynamic range, naturally capturing robust dynamic cues and suppressing static noise.<n>We propose textbfEventGait, an end-to-end dual-stream framework that separately models motion and shape while preserving the advantages of events.<n>Our approach sets a new state of the art on both synthesized and real-world event-based gait benchmarks, highlighting the robustness and potential of event-driven gait analysis.
  arXiv  Detail & Related papers  (2026-05-21T08:12:04Z)
• DarkShake-DVS: Event-based Human Action Recognition under Low-light andShaking Camera Conditions [31.62492338790292]
  Human Action Recognition (HAR) is a fundamental computer vision task with diverse real-world applications.<n>Current research faces two key challenges: absence of a benchmark integrating low-light conditions, 6-DoF motion, and synchronized IMU data.<n>We propose Event-IMU Stabilized HAR (EIS-HAR), with two modules.
  arXiv  Detail & Related papers  (2026-05-20T03:47:47Z)
• EPRBench: A High-Quality Benchmark Dataset for Event Stream Based Visual Place Recognition [54.55914886780534]
  Event stream-based Visual Place Recognition (VPR) is an emerging research direction that offers a compelling solution to the instability of conventional visible-light cameras under challenging conditions such as low illumination, overexposure, and high-speed motion.<n>We introduce EPRBench, a high-quality benchmark specifically designed for event stream-based VPR.<n>EPRBench comprises 10K event sequences and 65K event frames, collected using both handheld and vehicle-mounted setups to comprehensively capture real-world challenges across diverse viewpoints, weather conditions, and lighting scenarios.
  arXiv  Detail & Related papers  (2026-02-13T13:25:05Z)
• Event-based Facial Keypoint Alignment via Cross-Modal Fusion Attention and Self-Supervised Multi-Event Representation Learning [16.170645576584487]
  Event cameras offer unique advantages for facial keypoint alignment under challenging conditions.<n>We propose a novel framework based on cross-modal fusion attention (CMFA) and self-supervised multi-event representation learning (SSMER) for event-based facial keypoint alignment.
  arXiv  Detail & Related papers  (2025-09-29T16:00:50Z)
• Inter-event Interval Microscopy for Event Cameras [52.05337480169517]
  Event cameras, an innovative bio-inspired sensor, differ from traditional cameras by sensing changes in intensity rather than directly perceiving intensity.<n>We achieve event-to-intensity conversion using a static event camera for both static and dynamic scenes in fluorescence microscopy.<n>We have collected IEIMat dataset under various scenes including high dynamic range and high-speed scenarios.
  arXiv  Detail & Related papers  (2025-04-07T11:05:13Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• Multi-modal Multi-platform Person Re-Identification: Benchmark and Method [58.59888754340054]
  MP-ReID is a novel dataset designed specifically for multi-modality and multi-platform ReID.<n>This benchmark compiles data from 1,930 identities across diverse modalities, including RGB, infrared, and thermal imaging.<n>We introduce Uni-Prompt ReID, a framework with specific-designed prompts, tailored for cross-modality and cross-platform scenarios.
  arXiv  Detail & Related papers  (2025-03-21T12:27:49Z)
• Event-assisted Low-Light Video Object Segmentation [47.28027938310957]
  Event cameras offer promise in enhancing object visibility and aiding VOS methods under such low-light conditions. This paper introduces a pioneering framework tailored for low-light VOS, leveraging event camera data to elevate segmentation accuracy.
  arXiv  Detail & Related papers  (2024-04-02T13:41:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.