ActFER: Agentic Facial Expression Recognition via Active Tool-Augmented Visual Reasoning

• URL: http://arxiv.org/abs/2604.08990v1
• Date: Fri, 10 Apr 2026 05:53:19 GMT
• Title: ActFER: Agentic Facial Expression Recognition via Active Tool-Augmented Visual Reasoning
• Abstract summary: We propose ActFER, an agentic framework that reformulates FER as active visual evidence acquisition followed by multimodal reasoning.<n>We further develop Utility-Calibrated GRPO, a reinforcement learning algorithm tailored to agentic FER.
• Score: 55.562958315741646
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have created new opportunities for facial expression recognition (FER), moving it beyond pure label prediction toward reasoning-based affect understanding. However, existing MLLM-based FER methods still follow a passive paradigm: they rely on externally prepared facial inputs and perform single-pass reasoning over fixed visual evidence, without the capability for active facial perception. To address this limitation, we propose ActFER, an agentic framework that reformulates FER as active visual evidence acquisition followed by multimodal reasoning. Specifically, ActFER dynamically invokes tools for face detection and alignment, selectively zooms into informative local regions, and reasons over facial Action Units (AUs) and emotions through a visual Chain-of-Thought. To realize such behavior, we further develop Utility-Calibrated GRPO (UC-GRPO), a reinforcement learning algorithm tailored to agentic FER. UC-GRPO uses AU-grounded multi-level verifiable rewards to densify supervision, query-conditional contrastive utility estimation to enable sample-aware dynamic credit assignment for local inspection, and emotion-aware EMA calibration to reduce noisy utility estimates while capturing emotion-wise inspection tendencies. This algorithm enables ActFER to learn both when local inspection is beneficial and how to reason over the acquired evidence. Comprehensive experiments show that ActFER trained with UC-GRPO consistently outperforms passive MLLM-based FER baselines and substantially improves AU prediction accuracy.

Related papers

• ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuning [53.17987412112712]
  Reinforcement Learning (RL) has demonstrated significant potential for improving Vision-Language-Action (VLA) models on complex manipulation tasks.<n>In this work, we first investigate the exploration stagnation bottleneck in current VLA-RL frameworks.<n>We introduce RL Exploration Token (ExToken), a simple yet general framework that condition VLA policies on discrete behavioral priors.
  arXiv  Detail & Related papers  (2026-07-14T16:04:41Z)
• Seeing Isn't Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents [20.463811944310475]
  Estimate-Verify-Update (EVU) is a unified intervention mechanism that generates textual belief states explicitly.<n>EVU consistently yields substantial gains in task success rates.<n>Our approach effectively mitigates belief inertia, advancing the development of more robust embodied agents.
  arXiv  Detail & Related papers  (2026-04-19T04:36:33Z)
• Rethinking VLMs for Image Forgery Detection and Localization [55.32700985102152]
  We study how to fully leverage vision-language models (VLMs) to assist the image forgery detection and localization (IFDL) task.<n>Building on these findings, we propose a new IFDL pipeline named IFDL-VLM.<n>The experimental results show that we consistently achieve new state-of-the-art performance in detection, localization, and interpretability.
  arXiv  Detail & Related papers  (2026-03-13T12:21:31Z)
• From Intuition to Investigation: A Tool-Augmented Reasoning MLLM Framework for Generalizable Face Anti-Spoofing [47.26568822352609]
  We propose the Tool-Augmented Reasoning FAS (TAR-FAS) framework, which reformulates the Face Anti-Spoofing task as a Chain-of-Thought with Visual Tools (CoT-VT) paradigm.<n>We show that TAR-FAS achieves SOTA performance while providing fine-grained visual investigation for trustworthy spoof detection.
  arXiv  Detail & Related papers  (2026-03-01T10:28:35Z)
• Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation [51.743225614196774]
  Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language reasoning.<n>They remain vulnerable to hallucination, where generated content deviates from visual evidence.<n>Recent vision enhancement methods attempt to address this issue by reinforcing visual tokens during decoding.<n>We propose Adaptive Visual Reinforcement (AIR), a training-free framework for MLLMs.
  arXiv  Detail & Related papers  (2026-02-27T14:18:51Z)
• TAG: Thinking with Action Unit Grounding for Facial Expression Recognition [18.35567824514563]
  We propose TAG (Thinking with Action Unit Grounding), a vision--language framework that explicitly constrains multimodal reasoning to be supported by facial Action Units (AUs)<n> evaluated on RAF-DB, FERPlus, and AffectNet.
  arXiv  Detail & Related papers  (2026-02-21T09:00:52Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Meaningless Tokens, Meaningful Gains: How Activation Shifts Enhance LLM Reasoning [53.35553353785948]
  Motivated by the puzzling observation that inserting long sequences of meaningless tokens before the query prompt can consistently enhance reasoning LLM performance, this work analyzes the underlying mechanism driving this phenomenon.<n>We find that the improvements arise from a redistribution of activations in the LLM's layers, where near zero activations become less frequent while large magnitude activations increase.<n>We propose a lightweight inference-time technique that modifies activations directly without altering the input sequence.
  arXiv  Detail & Related papers  (2025-10-01T15:39:38Z)
• Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification [17.67273082468732]
  Verifiers -- functions assigning rewards to agent behavior -- have been key for AI progress in domains like math and board games.<n>We evaluate Multimodal Large Language Models (MLLMs) as verifiers of agent trajectories across web navigation, computer use, and robotic manipulation.<n>We propose Self-Grounded Verification (SGV), a lightweight method that enables more effective use of MLLMs' knowledge and reasoning.
  arXiv  Detail & Related papers  (2025-07-15T18:50:29Z)
• Controlling Large Language Model Agents with Entropic Activation Steering [20.56909601159833]
  We introduce Entropic Activation Steering (EAST), an activation steering method for in-context learning agents. We show that EAST can effectively manipulate an LLM agent's exploration by directly affecting the high-level actions parsed from the outputs of the LLM. We also reveal how applying this control modulates the uncertainty exhibited in the LLM's thoughts, guiding the agent towards more exploratory actions.
  arXiv  Detail & Related papers  (2024-06-01T00:25:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.