EmotionAI: A Privacy-Preserving Computational Intelligence Pipeline for Speech-Emotion-Grounded Conversational Analysis

• URL: http://arxiv.org/abs/2606.24941v1
• Date: Mon, 22 Jun 2026 20:45:49 GMT
• Title: EmotionAI: A Privacy-Preserving Computational Intelligence Pipeline for Speech-Emotion-Grounded Conversational Analysis
• Abstract summary: EmotionAI is a fully local Computational Intelligence pipeline that couples Speech Emotion Recognition with generative reasoning.<n>Speaker diarisation, Whisper Automatic Speech Recognition (ASR) and a wav2vec2 emotion classifier produce per-segment affective evidence.<n>The contribution is not state-of-the-art SER but an auditable, privacy-preserving integration of imperfect affective evidence into grounded conversational analysis.
• Score: 1.5803208833562954
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reviewing recorded interviews for affective cues such as composure, hesitation and agitation is slow and subjective, and cloud services that could automate it require sensitive audio to leave the device. EmotionAI is a fully local Computational Intelligence (CI) pipeline that couples Speech Emotion Recognition (SER) with generative reasoning. Speaker diarisation, Whisper Automatic Speech Recognition (ASR) and a wav2vec2 emotion classifier produce per-segment affective evidence, which is then passed to an adversarial three-model local Large Language Model (LLM) panel for timestamp-grounded and citation-constrained question answering. Zero-shot evaluation on the RAVDESS four-class English subset (n = 672) exposes cross-corpus fragility rather than classifier superiority: the deployed classifier scores 48.8% accuracy, above random (24.9%) and majority (28.6%) baselines but below an in-domain MFCC + logistic-regression comparator (71.0%). The complete pipeline runs in a mean 157 s on CPU (real-time factor approximately 1.33) with zero external calls. The contribution is not state-of-the-art SER but an auditable, privacy-preserving integration of imperfect affective evidence into grounded conversational analysis, together with an honest empirical account of where cross-corpus transfer and human-centred validation still fall short.

Related papers

• SISER: Speaker-Invariant Speech Emotion Recognition with Entropy-Based Adversarial Training [4.0913617519821255]
  Speech emotion recognition (SER) faces two fundamental challenges: scarcity of labeled data and inter-speaker variability.<n>We propose SISER (Speaker-Invariant Speech Emotion Recognition), integrating wav2vec 2.0 as a feature encoder and ECAPA-TDNN as a speaker discriminator.<n>SISER achieves a UA of 60.63%, outperforming the baseline (51.15%) and wav2vec 2.0 without speaker suppression (56.46%)
  arXiv  Detail & Related papers  (2026-08-31T16:48:43Z)
• Dual-Scale State-Space Modeling with Speaker-Wise Dynamic CRF for Speech Emotion Recognition in Conversation [1.6389263804357965]
  We propose DSSM-CRF, an audio-only architecture for emotion recognition.<n>Bidirectional state-space models encode fused self-supervised speech representations at frame and dialogue scales.<n> Consecutive utterances in a speaker's chain form a transition pair whose score combines a corpus-level transition matrix with a residual predicted from the two contextualized utterances.
  arXiv  Detail & Related papers  (2026-08-23T12:56:03Z)
• SpeechEQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models [42.27807245438137]
  We introduce textscSpeechEQ, a comprehensive framework designed to evaluate the sociolinguistic reasoning of Speech-Language Models (SLMs)<n>The framework includes a validated dataset of 2,265 dialogues across 15 Emotional Quotient (EQ) subscales grounded in EQ-i 2.0 theory.
  arXiv  Detail & Related papers  (2026-06-24T16:03:38Z)
• ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models [60.14219417402433]
  Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavior without conscious retrieval.<n>We introduce ImplicitMemBench, the first systematic benchmark evaluating implicit memory through three constructs drawn from standard cognitive-science accounts of non-declarative memory.<n>Our 300-item suite employs a unified Learning/Priming-Interfere-Test protocol with first-attempt scoring.
  arXiv  Detail & Related papers  (2026-04-09T10:26:32Z)
• VoxEmo: Benchmarking Speech Emotion Recognition with Speech LLMs [54.75016325571445]
  Speech Large Language Models (LLMs) show great promise for speech emotion recognition via generative interfaces.<n> shifting from closed-set to open text generation introduces zero-shotity, making evaluation highly sensitive to prompts.<n>We present VoxEmo, a comprehensive SER benchmark encompassing 35 emotion corpora across 15 languages for Speech LLMs.
  arXiv  Detail & Related papers  (2026-03-09T21:10:34Z)
• ADEPT: RL-Aligned Agentic Decoding of Emotion via Evidence Probing Tools -- From Consensus Learning to Ambiguity-Driven Emotion Reasoning [67.22219034602514]
  We introduce ADEPT (Agentic Decoding of Emotion via Evidence Probing Tools), a framework that reframes emotion recognition as a multi-turn inquiry process.<n> ADEPT transforms an SLLM into an agent that maintains an evolving candidate emotion set and adaptively invokes dedicated semantic and acoustic probing tools.<n>We show that ADEPT improves primary emotion accuracy in most settings while substantially improving minor emotion characterization.
  arXiv  Detail & Related papers  (2026-02-13T08:33:37Z)
• AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering [97.52852990265136]
  We introduce AQAScore, a backbone-agnostic evaluation framework that leverages the reasoning capabilities of audio-aware large language models.<n>We evaluate AQAScore across multiple benchmarks, including human-rated relevance, pairwise comparison, and compositional reasoning tasks.
  arXiv  Detail & Related papers  (2026-01-21T07:35:36Z)
• Understanding Emotion in Discourse: Recognition Insights and Linguistic Patterns for Generation [0.36980845568339205]
  We conduct a rigorous ablation study with 10-seed evaluation for recognition.<n>For linguistic analysis, we analyze 5,286 discourse marker occurrences.<n>We find a significant association between emotion and marker positioning.
  arXiv  Detail & Related papers  (2026-01-01T02:49:44Z)
• When Robots Should Say "I Don't Know": Benchmarking Abstention in Embodied Question Answering [27.058973883094]
  Embodied Question Answering (EQA) requires an agent to interpret language, perceive its environment, and navigate within 3D scenes to produce responses.<n>We focus on a minimal requirement for EQA agents, abstention: knowing when to withhold an answer.<n>We derive five representative categories requiring abstention: actionability, referential underspecification, preference dependence, information unavailability, and false presupposition.
  arXiv  Detail & Related papers  (2025-12-04T09:17:40Z)
• SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models [60.72029578488467]
  SpeechR is a unified benchmark for evaluating reasoning over speech in large audio-language models.<n>It evaluates models along three key dimensions: factual retrieval, procedural inference, and normative judgment.<n> Evaluations on eleven state-of-the-art LALMs reveal that high transcription accuracy does not translate into strong reasoning capabilities.
  arXiv  Detail & Related papers  (2025-08-04T03:28:04Z)
• Speech Emotion Detection Based on MFCC and CNN-LSTM Architecture [0.0]
  This paper processes the initial audio input into waveplot and spectrum for analysis and concentrates on multiple features including MFCC as targets for feature extraction.<n>The architecture achieved an accuracy of 61.07% comprehensively for the test set, among which the detection of anger and neutral reaches a performance of 75.31% and 71.70% respectively.
  arXiv  Detail & Related papers  (2025-01-18T06:15:54Z)
• EmoDiarize: Speaker Diarization and Emotion Identification from Speech Signals using Convolutional Neural Networks [0.0]
  This research explores the integration of deep learning techniques in speech emotion recognition. It introduces a framework that combines a pre-existing speaker diarization pipeline and an emotion identification model built on a Convolutional Neural Network (CNN) The proposed model yields an unweighted accuracy of 63%, demonstrating remarkable efficiency in accurately identifying emotional states within speech signals.
  arXiv  Detail & Related papers  (2023-10-19T16:02:53Z)
• Decoding speech perception from non-invasive brain recordings [48.46819575538446]
  We introduce a model trained with contrastive-learning to decode self-supervised representations of perceived speech from non-invasive recordings. Our model can identify, from 3 seconds of MEG signals, the corresponding speech segment with up to 41% accuracy out of more than 1,000 distinct possibilities.
  arXiv  Detail & Related papers  (2022-08-25T10:01:43Z)
• HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units [81.53783563025084]
  We propose an offline clustering step to provide aligned target labels for a BERT-like prediction loss. A key ingredient of our approach is applying the prediction loss over the masked regions only. HuBERT shows up to 19% and 13% relative WER reduction on the more challenging dev-other and test-other evaluation subsets.
  arXiv  Detail & Related papers  (2021-06-14T14:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.