EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech

• URL: http://arxiv.org/abs/2602.01170v1
• Date: Sun, 01 Feb 2026 11:38:30 GMT
• Title: EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech
• Abstract summary: EmoAra is an emotion-preserving pipeline for cross-lingual spoken communication.<n>It is motivated by banking customer service where emotional context affects service quality.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This work presents EmoAra, an end-to-end emotion-preserving pipeline for cross-lingual spoken communication, motivated by banking customer service where emotional context affects service quality. EmoAra integrates Speech Emotion Recognition, Automatic Speech Recognition, Machine Translation, and Text-to-Speech to process English speech and deliver an Arabic spoken output while retaining emotional nuance. The system uses a CNN-based emotion classifier, Whisper for English transcription, a fine-tuned MarianMT model for English-to-Arabic translation, and MMS-TTS-Ara for Arabic speech synthesis. Experiments report an F1-score of 94% for emotion classification, translation performance of BLEU 56 and BERTScore F1 88.7%, and an average human evaluation score of 81% on banking-domain translations. The implementation and resources are available at the accompanying GitHub repository.

Related papers

• Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars [0.0]
  NEST-V1 is a proof-of-concept framework that generates emotion-conditioned Nepali Sign Language avatars from spoken input.<n>Our preliminary results indicate the viability of real-time, emotionally expressive sign language communication systems for the hearing-impaired community.
  arXiv  Detail & Related papers  (2026-05-04T15:17:49Z)
• Hybrid CNN-Transformer Architecture for Arabic Speech Emotion Recognition [0.0]
  We present an Arabic Speech Emotion Recognition (SER) system based on a hybrid CNN-Transformer architecture.<n>The proposed model achieved 97.8% accuracy and a macro F1-score of 0.98.
  arXiv  Detail & Related papers  (2026-03-28T16:11:56Z)
• Speaker Style-Aware Phoneme Anchoring for Improved Cross-Lingual Speech Emotion Recognition [58.74986434825755]
  Cross-lingual speech emotion recognition is a challenging task due to differences in phonetic variability and speaker-specific expressive styles.<n>We propose a speaker-style aware phoneme anchoring framework that aligns emotional expression at the phonetic and speaker levels.<n>Our method builds emotion-specific speaker communities via graph-based clustering to capture shared speaker traits.
  arXiv  Detail & Related papers  (2025-09-19T21:03:21Z)
• EmoTale: An Enacted Speech-emotion Dataset in Danish [4.228593407506635]
  EmoTale is a corpus of Danish and English speech recordings with their associated enacted emotion annotations.<n>We develop SER models for EmoTale and the reference datasets using self-supervised speech model embeddings and the openSMILE feature extractor.<n>The best model achieves an unweighted average recall (UAR) of 64.1% on the EmoTale corpus.
  arXiv  Detail & Related papers  (2025-08-20T09:01:54Z)
• HausaNLP at SemEval-2025 Task 11: Hausa Text Emotion Detection [0.0]
  This paper presents our approach to multi-label emotion detection in Hausa, a low-resource African language, for SemEval Track A.<n>We fine-tuned AfriBERTa, a transformer-based model pre-trained on African languages, to classify Hausa text into six emotions: anger, disgust, fear, joy, sadness, and surprise.
  arXiv  Detail & Related papers  (2025-06-19T15:19:35Z)
• EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic [0.021665899581403608]
  This research introduces a bilingual dataset comprising 23,456 entries for Arabic and 10,036 entries for English, annotated for emotions and hope speech.<n>The dataset provides comprehensive annotations capturing emotion intensity, complexity, and causes, alongside detailed classifications and subcategories for hope speech.
  arXiv  Detail & Related papers  (2025-05-17T11:21:58Z)
• EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting [48.56693150755667]
  We propose EmoVoice, a novel emotion-controllable TTS model.<n>EmoVoice exploits large language models (LLMs) to enable fine-grained freestyle natural language emotion control.<n>EmoVoice achieves state-of-the-art performance on the English EmoVoice-DB test set.
  arXiv  Detail & Related papers  (2025-04-17T11:50:04Z)
• Usefulness of Emotional Prosody in Neural Machine Translation [1.0205541448656992]
  We propose to improve translation quality by adding another external source of information: the automatically recognized emotion in the voice. This work is motivated by the assumption that each emotion is associated with a specific lexicon that can overlap between emotions. We show that integrating emotion information, especially arousal, into NMT systems leads to better translations.
  arXiv  Detail & Related papers  (2024-04-27T18:04:28Z)
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation [90.71078166159295]
  We introduce SeamlessM4T, a single model that supports speech-to-speech translation, speech-to-text translation, text-to-text translation, and automatic speech recognition for up to 100 languages. We developed the first multilingual system capable of translating from and into English for both speech and text. On FLEURS, SeamlessM4T sets a new standard for translations into multiple target languages, achieving an improvement of 20% BLEU over the previous SOTA in direct speech-to-text translation.
  arXiv  Detail & Related papers  (2023-08-22T17:44:18Z)
• MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition [51.412413996510814]
  We propose MixSpeech, a cross-modality self-learning framework that utilizes audio speech to regularize the training of visual speech tasks. MixSpeech enhances speech translation in noisy environments, improving BLEU scores for four languages on AVMuST-TED by +1.4 to +4.2.
  arXiv  Detail & Related papers  (2023-03-09T14:58:29Z)
• EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model [56.75775793011719]
  We introduce and publicly release a Mandarin emotion speech dataset including 9,724 samples with audio files and its emotion human-labeled annotation. Unlike those models which need additional reference audio as input, our model could predict emotion labels just from the input text and generate more expressive speech conditioned on the emotion embedding. In the experiment phase, we first validate the effectiveness of our dataset by an emotion classification task. Then we train our model on the proposed dataset and conduct a series of subjective evaluations.
  arXiv  Detail & Related papers  (2021-06-17T08:34:21Z)
• Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training [91.95855310211176]
  Emotional voice conversion aims to change the emotional state of an utterance while preserving the linguistic content and speaker identity. We propose a novel 2-stage training strategy for sequence-to-sequence emotional voice conversion with a limited amount of emotional speech data. The proposed framework can perform both spectrum and prosody conversion and achieves significant improvement over the state-of-the-art baselines in both objective and subjective evaluation.
  arXiv  Detail & Related papers  (2021-03-31T04:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.