Distance-aware Soft Prompt Learning for Multimodal Valence-Arousal Estimation

• URL: http://arxiv.org/abs/2603.13415v1
• Date: Thu, 12 Mar 2026 15:33:17 GMT
• Title: Distance-aware Soft Prompt Learning for Multimodal Valence-Arousal Estimation
• Abstract summary: We propose a novel framework for Valence-arousal (VA) estimation.<n>We partition the VA space into a 3X3 grid, defining nine emotional regions.<n>We employ a Gaussian kernel to compute soft labels based on the Euclidean distance between the ground truth coordinates and the region centers.
• Score: 2.8037951156321377
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Valence-arousal (VA) estimation is crucial for capturing the nuanced nature of human emotions in naturalistic environments. While pre-trained Vision-Language models like CLIP have shown remarkable semantic alignment capabilities, their application in continuous regression tasks is often limited by the discrete nature of text prompts. In this paper, we propose a novel multimodal framework for VA estimation that introduces Distance-aware Soft Prompt Learning to bridge the gap between semantic space and continuous dimensions. Specifically, we partition the VA space into a 3X3 grid, defining nine emotional regions, each associated with distinct textual descriptions. Rather than a hard categorization, we employ a Gaussian kernel to compute soft labels based on the Euclidean distance between the ground truth coordinates and the region centers, allowing the model to learn fine-grained emotional transitions. For multimodal integration, our architecture utilizes a CLIP image encoder and an Audio Spectrogram Transformer (AST) to extract robust spatial and acoustic features. These features are temporally modeled via Gated Recurrent Units (GRUs) and integrated through a hierarchical fusion scheme that sequentially combines cross-modal attention for alignment and gated fusion for adaptive refinement. Experimental results on the Aff-Wild2 dataset demonstrate that our proposed semantic-guided approach significantly enhances the accuracy of VA estimation, achieving competitive performance in unconstrained ``in-the-wild'' scenarios.

Related papers

• Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding [63.599342750566485]
  SPAR is a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation.<n>Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively.
  arXiv  Detail & Related papers  (2026-08-29T09:58:24Z)
• MG-RWKV: Multi-Grained Context-Aware RWKV for Temporal Forgery Localization [43.92649543343891]
  MG-RWKV is a multi-granularity framework for full-sequence processing with O(T) complexity.<n>Our framework features three core innovations: (1) a Bidirectional RWKV architecture that captures bidirectional temporal contexts without quadratic overhead; (2) a Multi-Granularity Mixture of Experts (MG-MoE) that performs dynamic routing over explicit temporal receptive fields, adaptively selecting granularities based on forgery duration to significantly enhance decision interpretability; and (3) Cross-Granularity Consistency (CGC) which aligns adjacent feature pyramid levels through hierarchical scale-wise pairing and spatial boundary-aware weighting, effectively reducing false positives
  arXiv  Detail & Related papers  (2026-07-01T13:04:35Z)
• OmniGF: A Dual-Branch Vision-Language Framework for Unified Gaze Following [59.53720386342017]
  We propose a unified vision-language framework that adapts foundational Vision-Language Models for scalable multi-person gaze reasoning.<n>By modeling all individuals, OmniGF seamlessly integrates precise spatial gaze target estimation, semantic gaze prediction, and complex social gaze reasoning.
  arXiv  Detail & Related papers  (2026-05-26T00:08:06Z)
• AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding [8.941979904606308]
  Vision-Language Models (VLMs) have enabled autonomous GUI agents that translate natural language instructions into executable screen coordinates.<n>Existing zoom-in strategies rely on fixed anchors, grids, or reinforcement learning.<n>We propose AutoFocus, a training-free, uncertainty-aware active visual search framework for GUI grounding.
  arXiv  Detail & Related papers  (2026-05-04T14:18:46Z)
• ConInfer: Context-Aware Inference for Training-Free Open-Vocabulary Remote Sensing Segmentation [12.428190609914594]
  Training-free open-vocabulary remote sensing segmentation (OVRSS) has emerged as a promising paradigm for achieving category-agnostic semantic understanding in remote sensing imagery.<n>We propose ConInfer, a context-aware inference framework for OVRSS that performs joint prediction across multiple spatial units.<n>Our method significantly enhances segmentation consistency, robustness, and generalization in complex remote sensing environments.
  arXiv  Detail & Related papers  (2026-03-31T05:12:02Z)
• Thinking with Images as Continuous Actions: Numerical Visual Chain-of-Thought [55.65577137924979]
  We propose a framework that enables MLLMs to reason over images using continuous numerical coordinates.<n> NV-CoT expands the MLLM action space from discrete vocabulary tokens to a continuous Euclidean space.<n>Experiments on three benchmarks demonstrate that NV-CoT significantly improves localization precision and final answer accuracy.
  arXiv  Detail & Related papers  (2026-02-27T12:04:07Z)
• Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scoping [61.459927600301654]
  Multi-condition control is bottlenecked by the conventional concatenate-and-attend'' strategy.<n>Our analysis reveals that much of this cross-modal interaction is spatially or semantically redundant.<n>We propose Position-aligned and Keyword-scoped Attention (PKA), a highly efficient framework designed to eliminate these redundancies.
  arXiv  Detail & Related papers  (2026-02-06T16:39:10Z)
• RSGround-R1: Rethinking Remote Sensing Visual Grounding through Spatial Reasoning [61.84363374647606]
  Remote Sensing Visual Grounding (RSVG) aims to localize target objects in large-scale aerial imagery based on natural language descriptions.<n>These descriptions often rely heavily on positional cues, posing unique challenges for Multimodal Large Language Models (MLLMs) in spatial reasoning.<n>We propose a reasoning-guided, position-aware post-training framework, dubbed textbfRSGround-R1, to progressively enhance spatial understanding.
  arXiv  Detail & Related papers  (2026-01-29T12:35:57Z)
• Fourier-RWKV: A Multi-State Perception Network for Efficient Image Dehazing [26.57698394898644]
  We propose a novel dehazing framework based on a Multi-State Perception paradigm.<n>Fourier-RWKV delivers state-of-the-art performance across diverse haze scenarios.
  arXiv  Detail & Related papers  (2025-12-09T01:35:56Z)
• Few-Shot Remote Sensing Image Scene Classification with CLIP and Prompt Learning [0.9558392439655014]
  We explore prompt learning as a lightweight and efficient adaptation strategy for few-shot remote sensing image scene classification.<n>We benchmark these prompt-learning methods against two standard baselines: zero-shot CLIP with hand-crafted prompts and a linear probe trained on frozen CLIP features.<n>Our findings underscore prompt learning as a scalable and efficient solution for bridging the domain gap in satellite and aerial imagery.
  arXiv  Detail & Related papers  (2025-10-28T11:39:22Z)
• From Points to Spheres: A Geometric Reinterpretation of Variational Autoencoders [0.0]
  Variational Autoencoder is typically understood from the perspective of probabilistic inference.<n>In this work, we propose a new geometric reinterpretation which complements the probabilistic view and enhances its intuitiveness.
  arXiv  Detail & Related papers  (2025-07-23T06:52:00Z)
• econSG: Efficient and Multi-view Consistent Open-Vocabulary 3D Semantic Gaussians [56.85804719947]
  We propose econSG for open-vocabulary semantic segmentation with 3DGS.<n>Our econSG shows state-of-the-art performance on four benchmark datasets compared to the existing methods.
  arXiv  Detail & Related papers  (2025-04-08T13:12:31Z)
• EgoSplat: Open-Vocabulary Egocentric Scene Understanding with Language Embedded 3D Gaussian Splatting [108.15136508964011]
  EgoSplat is a language-embedded 3D Gaussian Splatting framework for open-vocabulary egocentric scene understanding.<n>EgoSplat achieves state-of-the-art performance in both localization and segmentation tasks on two datasets.
  arXiv  Detail & Related papers  (2025-03-14T12:21:26Z)
• FER-former: Multi-modal Transformer for Facial Expression Recognition [14.219492977523682]
  A novel multifarious supervision-steering Transformer for Facial Expression Recognition is proposed in this paper. Our approach features multi-granularity embedding integration, hybrid self-attention scheme, and heterogeneous domain-steering supervision. Experiments on popular benchmarks demonstrate the superiority of the proposed FER-former over the existing state-of-the-arts.
  arXiv  Detail & Related papers  (2023-03-23T02:29:53Z)
• Improve Variational Autoencoder for Text Generationwith Discrete Latent Bottleneck [52.08901549360262]
  Variational autoencoders (VAEs) are essential tools in end-to-end representation learning. VAEs tend to ignore latent variables with a strong auto-regressive decoder. We propose a principled approach to enforce an implicit latent feature matching in a more compact latent space.
  arXiv  Detail & Related papers  (2020-04-22T14:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.