Investigation for Relative Voice Impression Estimation

• URL: http://arxiv.org/abs/2602.14172v2
• Date: Wed, 18 Feb 2026 01:49:29 GMT
• Title: Investigation for Relative Voice Impression Estimation
• Authors: Kenichi Fujita, Yusuke Ijima,
• Abstract summary: This study investigates relative voice impression estimation (RIE)<n>RIE is a framework for predicting the perceptual difference between two utterances from the same speaker.<n>We compare three modeling approaches: classical acoustic features commonly used for speech emotion recognition, self-supervised speech representations, and multimodal large language models.
• Score: 16.024524623990484
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Paralinguistic and non-linguistic aspects of speech strongly influence listener impressions. While most research focuses on absolute impression scoring, this study investigates relative voice impression estimation (RIE), a framework for predicting the perceptual difference between two utterances from the same speaker. The estimation target is a low-dimensional vector derived from subjective evaluations, quantifying the perceptual shift of the second utterance relative to the first along an antonymic axis (e.g., ``Dark--Bright''). To isolate expressive and prosodic variation, we used recordings of a professional speaker reading a text in various styles. We compare three modeling approaches: classical acoustic features commonly used for speech emotion recognition, self-supervised speech representations, and multimodal large language models (MLLMs). Our results demonstrate that models using self-supervised representations outperform methods with classical acoustic features, particularly in capturing complex and dynamic impressions (e.g., ``Cold--Warm'') where classical features fail. In contrast, current MLLMs prove unreliable for this fine-grained pairwise task. This study provides the first systematic investigation of RIE and demonstrates the strength of self-supervised speech models in capturing subtle perceptual variations.

Related papers

• On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation [88.77441715819366]
  Generative spoken language models pretrained on large-scale raw audio can continue a speech prompt with appropriate content.<n>We propose a variety of likelihood- and generative-based evaluation methods that serve in place of naive global token perplexity.
  arXiv  Detail & Related papers  (2026-01-09T22:01:56Z)
• ELEGANCE: Efficient LLM Guidance for Audio-Visual Target Speech Extraction [88.41471266579333]
  We propose ELEGANCE, a novel framework that incorporates linguistic knowledge from large language models (LLMs) into AV-TSE models.<n> Comprehensive experiments with RoBERTa, Qwen3-0.6B, and Qwen3-4B on two AV-TSE backbones show significant improvements.
  arXiv  Detail & Related papers  (2025-11-09T08:50:11Z)
• SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models [60.72029578488467]
  SpeechR is a unified benchmark for evaluating reasoning over speech in large audio-language models.<n>It evaluates models along three key dimensions: factual retrieval, procedural inference, and normative judgment.<n> Evaluations on eleven state-of-the-art LALMs reveal that high transcription accuracy does not translate into strong reasoning capabilities.
  arXiv  Detail & Related papers  (2025-08-04T03:28:04Z)
• Affect Models Have Weak Generalizability to Atypical Speech [6.392336908224424]
  We evaluate models for recognizing categorical and dimensional affect from speech on a dataset of atypical speech.<n>We find that the output of affect models is significantly impacted by the presence and degree of speech atypicalities.
  arXiv  Detail & Related papers  (2025-04-22T21:40:17Z)
• Speechworthy Instruction-tuned Language Models [71.8586707840169]
  We show that both prompting and preference learning increase the speech-suitability of popular instruction-tuned LLMs. We share lexical, syntactical, and qualitative analyses to showcase how each method contributes to improving the speech-suitability of generated responses.
  arXiv  Detail & Related papers  (2024-09-23T02:34:42Z)
• Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis [6.382013662443799]
  We used multi-modal fusion representations from pre-trained models to generate state-of-the-art speech emotion estimation. We discovered that lexical representations are more robust to distortions compared to acoustic representations.
  arXiv  Detail & Related papers  (2023-03-03T18:22:32Z)
• Pre-Finetuning for Few-Shot Emotional Speech Recognition [20.894029832911617]
  We view speaker adaptation as a few-shot learning problem. We propose pre-finetuning speech models on difficult tasks to distill knowledge into few-shot downstream classification objectives.
  arXiv  Detail & Related papers  (2023-02-24T22:38:54Z)
• Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation [6.382013662443799]
  Estimating dimensional emotions from acoustic speech signals is challenging. We show that pre-trained acoustic models can improve valence estimation from speech. We report new state-of-the-art "text-free" acoustic-only dimensional emotion estimation.
  arXiv  Detail & Related papers  (2022-07-02T17:34:44Z)
• Self-supervised models of audio effectively explain human cortical responses to speech [71.57870452667369]
  We capitalize on the progress of self-supervised speech representation learning to create new state-of-the-art models of the human auditory system. We show that these results show that self-supervised models effectively capture the hierarchy of information relevant to different stages of speech processing in human cortex.
  arXiv  Detail & Related papers  (2022-05-27T22:04:02Z)
• Unsupervised Personalization of an Emotion Recognition System: The Unique Properties of the Externalization of Valence in Speech [37.6839508524855]
  Adapting a speech emotion recognition system to a particular speaker is a hard problem, especially with deep neural networks (DNNs) This study proposes an unsupervised approach to address this problem by searching for speakers in the train set with similar acoustic patterns as the speaker in the test set. We propose three alternative adaptation strategies: unique speaker, oversampling and weighting approaches.
  arXiv  Detail & Related papers  (2022-01-19T22:14:49Z)
• Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue [76.88174667929665]
  A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. In the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely. We propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history.
  arXiv  Detail & Related papers  (2020-09-14T15:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.